Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on
Slack,
GitHub, Stack Overflow, and Reddit.
Wait, what’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
[@portabletext/react] Unknown block type "externalImage", specify a component for it in the `components.types` prop
Ok, let’s dive into this week’s tips and tricks!
Sending grouped tasks to CVAT
Community Slack member Daniel Fortunato asked,
“I am using FiftyOne to manage a grouped dataset with three images per group. I want to send these images to CVAT for annotation. How can I import all the images in each group as a single task with multiple jobs, rather than multiple tasks with one job each?
If each group slice has the same media type, e.g. images, then you can do this by selecting the desired group slices with select_group_slices(), and grouping by the group id with group_by(). Calling select_group_slices() will return a collection of images (or more generally, a view whose media_type attribute is the same as the group slices, rather than media_type = “group”. The following group_by() operation will rearrange the resulting DatasetView such that samples from the same group are located at neighboring indices.
Once this view has been created, it can be sent to CVAT using the annotate() method, with the segment_size argument set equal to the number of images per task — here the number of selected group slices.
Putting it all together, we can see what this would look like for the
Quickstart Groups dataset:
Linking detections on multiple samples
Community Slack member Marijn Lems asked,
“Is it possible to relate multiple detections on different samples in FiftyOne?”
In computer vision it is often the case that multiple images or media files have objects that are related to each other. For instance, in a dataset containing faces, such as
Families in the Wild, there could be multiple images that feature the same person’s face. Alternatively, in a grouped dataset like the
KITTI Multiview dataset, an image and a point-cloud could feature different representations of the same object.
One way to associate these detections is by creating a Detection-level metadata field that you populate with a unique identifier for each associated group of detections. Working with the
Quickstart Groups dataset, for example, linking a ground truth detection in the “left” and “right” images in the first group can be done as follows:
Merging large datasets into grouped dataset
Community Slack member Oğuz Hanoğlu asked,
“I have two large image datasets which I want to merge into a grouped dataset according to their values in a certain field. What is the most efficient way to do this?”
When dealing with large datasets, you want to perform as many operations in-database as possible to avoid the long times typical of loading large datasets into memory.
One way to do this is to create a temporary clone of each source dataset, then populate a Group field on each sample in these “single slice” datasets, and finally merge these datasets into a single grouped dataset. To merge datasets ds_left and ds_right based on their values in field my_field, this would look like:
This approach does require the set_values() method rather than the set_field() method, as the Group instances need to be instantiated in memory in order to guarantee that they have unique id’s . However, for large datasets this approach is far more efficient than an approach which iterates through samples in the datasets to be merged.
Annotating tagged labels from multiple fields
Community Slack member Daniel Bourke asked,
“I want to send a collection of the worst predictions to Label Studio for annotation, including both ground truth and prediction label fields. How do I do this, when the label schema for Label Studio only accepts one field?”
One way that you could do this now is to combine your incorrect ground truth and predictions into a single label field in FiftyOne first, then annotate that field in
Label Studio (since it’s now just one field to re-annotate).
First, you can clone the “ground_truth” and “predictions” fields into new fields. Then, you can filter on these fields to create a view containing only the samples of interest. And finally, you can use the merge_labels() method to merge one, temporary, cloned field into the other cloned field:
This consolidates your desired information into a single field so that it is suitable for sending to Label Studio. Additionally, it preserves the original fields.
Visualizing point clouds in the FiftyOne App
Community Slack member Marijn Lems asked,
“I have a dataset consisting of point-cloud data and I’d love to visualize this data in the FiftyOne App. How can I do this?”
In FiftyOne, you can define, populate, and perform operations on a point-cloud only dataset, but you will not be able to visualize this out-of-the-box without having an image or other media file associated with each point cloud. This is because point-clouds trigger FiftyOne’s
3d visualizer plugin, and it would be quite compute-intensive to have the sample-grid running a large number of these 3d visualizers at once.
One solution to this is to create a
Grouped Dataset with the point-cloud media as one slice, and to generate an image for each point-cloud, saving these images into another group slice. As an example, you could generate a bird’s eye view image for the point clouds in the collection
pcd_samples:
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
Join the FiftyOne community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
What’s next?
