FiftyOne Computer Vision Tips and Tricks – Nov 17, 2023

Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit.

As an open source community, the FiftyOne community is open to all. This means everyone is welcome to ask questions, and everyone is welcome to answer them. Continue reading to see the latest questions asked and answers provided!

Wait, what’s FiftyOne?

FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.

• If you like what you see on GitHub, give the project a star.
• Get started! We’ve made it easy to get up and running in a few minutes.
• Join the FiftyOne Slack community, we’re always happy to help.

Ok, let’s dive into this week’s tips and tricks!

Working with video object tracking

Community Slack member Rustem asked:

In regards to video object tracking, is there a way in FiftyOne to visualize and evaluate tracks?

Take a look at the FiftyOne quickstart video dataset, which consists of 10 video segments with dense object detections generated by human annotators. In it you’ll notice that the object identities are preserved through the index parameter. Also, check out to_trajectories(). You can use trajectory views to create views into your video datasets that contain one sample per each unique object trajectory defined by their (label, index) in a frame-level Detections or Polylines field.

Visualizing CVAT annotated skeletons

Community Slack member Queenie asked:

I used skeletons to annotate objects on CVAT. How can I add skeleton annotation data to the samples output so I can visualize the labels and skeleton in FiftyOne?

Check out the blog, “Creating Pose Skeletons from Scratch.” In the blog the following topics are covered:

• What are pose skeletons?
• Preparing the dataset
• Annotating the skeleton in CVAT
• Loading and visualizing the annotations in FiftyOne

Annotations is CVAT

Visualizing the annotations in FiftyOne

Computing embeddings on video frames

Community Slack member Dan asked:

Is there a way to compute the embeddings of the video frames if I have a video dataset that uses the compute_embeddings() function?

You can use:

frames = dataset.to_frames(sample_frames=True)

…to create a view of the frames of your video and get access to image functions like compute embeddings. For more information about the frames class check out the FiftyOne Docs. For tips on how to use embeddings, check out this blog and this tutorial in the FiftyOne Docs.

Geospatial queries in FiftyOne

Community Slack member Manikandan asked:

I have samples with a GeoLocation field containing GPS coordinates. Is it possible to use the FiftyOne App or the SDK to query the field? For example after importing fogeojson:

gps_coordinates = [163.7, 89.0]
fo_gps_result = fogeojson.parse_point(arg=gps_coordinates)
sample['gps'] = fo.GeoLocation.from_geo_json(fo_gps_result)

Although FiftyOne’s GeoLocation field only stores longitude and latitude, you can perform some basic queries based on those fields. There are also two functions you can use with geospatial data:  

• geo_near()– Sorts the samples in the collection by their proximity to a specified geolocation.
• geo_within()– Filters the samples in this collection to only include samples whose geolocation is within a specified boundary.

Learn more about geospatial support in the FiftyOne Docs.

Importing videos annotated in JSON with Labelbox

Community Slack member Yashovardhan asked:

Is there a way to import videos with JSON file annotations that were created using Labelbox? with a width and height of 720 and 1280:

Yes! Check out the fiftyone.utils.labelbox.import_from_labelbox utility method that enables you to import Labelbox videos and their annotation JSON into FiftyOne.