FiftyOne Computer Vision Tips and Tricks — Nov 4, 2022

Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit.

Wait, what’s FiftyOne?

FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.

• If you like what you see on GitHub, give the project a star
• Get started! We’ve made it easy to get up and running in a few minutes
• Join the FiftyOne Slack community, we’re always happy to help

Ok, let’s dive into this week’s tips and tricks!

Why aren’t my objects appearing in the App?

Community Slack member Matthew Millendorf asked,

“My Detections are not showing up in the FiftyOne App. Any suggestions on where the issue lies in the formatting of my data?”

<Sample: {
    'id': None,
    'media_type': 'image',
    'filepath': <filepath>, 
    'tags': [],
    'metadata': <ImageMetadata: {
        'size_bytes': 48271,
        'mime_type': 'image/jpeg',
        'width': 640,
        'height': 512,
        'num_channels': 3,
    }>,
    'inspection_id': <inspection_id>,
    'ground_truth': <Detections: {
        'detections': BaseList([
            <Detection: {
                'id': '6349b3472839e7d62bf3799b',
                'attributes': BaseDict({}),
                'tags': BaseList([]),
                'label': 'ModuleImage',
                'bounding_box': BaseList([320, 40, 15, 33]),
                'mask': None,
                'confidence': None,
                'index': None,
            }>,

FiftyOne detections use relative coordinates, so you’ll need to update the bounding box coordinates to be within [0, 1] x [0, 1] by dividing by the image height and width.

Learn more about Detections in the FiftyOne Docs.

Removing attributes when exporting a dataset

Community Slack member Xuân Huy Nguyễn asked,

“When exporting a dataset, some CVAT attributes are also exported which I don’t need. How can I remove these attributes when I run an export?”

There are a few options worth exploring depending on your use case. Taking COCO as an example, there is an optional extra_attrs=False parameter you can pass when exporting so as not to include custom attributes in the export. Using COCO as an example:

dataset.export(
    ...,
    dataset_type=fo.types.COCODetectionDataset,
    extra_attrs=False,
    ...,
)

Learn more about the COCO exporter and its available parameters in the FiftyOne Docs.

Another option is to create a view that excludes the unwanted attributes from your detections, then export that view to CVAT format.

view = dataset.exclude_fields([
    "your_field.detections.bad_attr1",
    "your_field.detections.bad_attr2",
    ...
])

view.export(
    ...,
    dataset_type=fo.types.COCODetectionDataset,
    ...,
)

The final option is to use the CVAT integration directly for your annotation tasks.

Overwriting a dataset

Community Slack member Oğuz Hanoğlu asked,

“What is the best way to overwrite a dataset?”

Two options. One approach is to first delete the existing dataset and then create the new dataset:

dataset_name = dataset.name
dataset.delete()

dataset = fo.Dataset(dataset_name)

Or you can combine these operations into a single step by passing the optional overwrite argument to the Dataset constructor:

dataset = fo.Dataset(dataset.name, overwrite=True)

Learn more about common and custom formats in the FiftyOne Docs.

Clipping bounding boxes to be within the image area only

Community Slack member Patrick Rowsome asked,

“I have an issue with an object detection dataset where some of the ground truth annotations are outside of the image just slightly, maybe 1–3 px. Is there a convenient way in FiftyOne to clip these bounding boxes to be within the image area only?”

If the bounding box coordinates are outside of 0–1, then you should clip them to that range. The easiest way to do this is to just loop your dataset and make the changes in a Python loop.

for sample in dataset.iter_samples(autosave=True):
    for detection in sample[label_field].detections:
        tlx1, tly1, w1, h1 = detection.bounding_box
        tlx2 = max(min(tlx1, 1), 0)
        w2 = w1 - (tlx2-tlx1)

        tly2 = max(min(tly1, 1), 0)
        h2 = h1 - (tly2-tly1)

        w2 = max(min(w2+tlx2, 1), 0) - tlx2
        h2 = max(min(h2+tly2, 1), 0) - tly2
        
        detection["bounding_box"] = [tlx2, tly2, w2, h2]

Note, the autosave context lets you avoid needing to call sample.save() each time.

Support for visualizing 2D point clouds

Community Slack member Matthew Millendorf asked,

“I am working on a geospatial point cloud registration problem, curious can you use FiftyOne for visualizing 2D point clouds where the points are lat/lon or UTM?

Yes, as of version 0.17, when you load a dataset in the App that contains a GeoLocation field with point data populated, you can open the Map tab to visualize a scatterplot of the location data:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-geo")

session = fo.launch_app(dataset)

What’s next?

• If you like what you see on GitHub, give the project a star
• Get started! We’ve made it easy to get up and running in a few minutes
• Join the FiftyOne Slack community, we’re always happy to help