FiftyOne Computer Vision Tips and Tricks — Oct 28, 2022

Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit.

Wait, what’s FiftyOne?

FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.

If you like what you see on GitHub, give the project a star
Get started! We’ve made it easy to get up and running in a few minutes
Join the FiftyOne Slack community, we’re always happy to help

Ok, let’s dive into this week’s tips and tricks!

Counting bounding boxes and visualizing anchor boxes

Community Slack member Sidney Guaro asked a two part question,

“How do you count the bounding boxes at different area scales? (e.g.area_small, area_medium, area_large) And is it possible to visualize the anchor boxes?”

If you need to filter detections by the bounding box area use ViewExpression You can visualize any number of sets of boxes you like if you extract the relevant information from your model and add them as Detectionsto your FiftyOne dataset.

Learn more about the ViewExpressionand adding object detections to a dataset in the FiftyOne Docs.

Computing embeddings on labels

Community Slack member Nadav Ben-Haim asked,

“Is it possible to compute embeddings on labels rather than images? Referencing this tutorial, it appears you can do it on full images, but I am interested in computing such embeddings on crops from bounding box labels.”

Yes! You can pass in the patches_fieldparameter to the visualization and the similarity functions available in FiftyOne Brain to use object-level embeddings rather than image-level.

Learn more about FiftyOne Brain in the FiftyOne Docs.

Working with features metadata

Community Slack member Adrian Loy asked,

“Can I see the distributions of metadata (e.g. frame count) features, similar to how I can see the distribution over Labels?”

Yes! You can easily pull out any specific metadata fields of interest and add them as their own field with the following:

dataset.set_values(“width”, dataset.values(“metadata.width”))

Alternatively, check out some of our Plotly integrations to see how to create custom plots on any fields of interest. (Note: If you work in notebooks, then they can also be interactive! But we also plan to add plugins to do this right in the App. Stay tuned!)

Remapping labels

Community Slack member Patrick Rowsome asked,

“I am trying to merge datasets and would like to map class names to be the same. For example, one dataset uses the person label and another uses pedestrian, ideally I would like to map all instances of pedestrians to the person label.”

Use the map_labels()function. It lets you create a view where labels are remapped, but if you want to make it permanent you can just call view.save()

Learn more about the map_labels() function in the FiftyOne Docs.

Matching tags within a Detections field

Community Slack member Geoffrey Keating asked,

“What would be the best way to match tags within a fo.Detectionsfield? I have individual detections tagged and would like to recall them.”

Try select_labels()which selects only the specified labels from the collection, with the returned view omitting samples, sample fields, and individual labels that do not match the specified selection criteria.

Alternatively, you can always write a view expression with filter_labels():

from fiftyone import ViewField as F

view = dataset.filter_labels("detections_field", F("tags").contains("my-tag"))

What’s next?

If you like what you see on GitHub, give the project a star
Get started! We’ve made it easy to get up and running in a few minutes
Join the FiftyOne Slack community, we’re always happy to help

Talk to a computer vision expert