Welcome to our weekly FiftyOne tips and tricks blog where we give practical pointers for using FiftyOne on topics inspired by discussions in the open source community. This week we’ll cover view stages.
Wait, What’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
- If you like what you see on GitHub, give the project a star.
- Get started! We’ve made it easy to get up and running in a few minutes.
- Join the FiftyOne Slack community, we’re always happy to help.
Ok, let’s dive into this week’s tips and tricks!
A primer on view stages
In FiftyOne, a ViewStage represents a stage of logic such as matching, filtering, or slicing, which can be used to determine the samples that appear in a dataset view. These stages can be chained together to produce flexible logical pipelines, enabling the programmatic isolation and evaluation of subsets of computer vision data. Every view stage is exposed via an associated method on Dataset and DatasetView instances.
Continue reading for some tips and tricks to help you master view stages in FiftyOne!
See what view stages have been applied
Many computer vision workflows involve chaining view stages together to create several tailored views of a dataset. If you ever lose track of what logical operations have been performed to generate a DatasetView, you can get a summary of this information by printing out the dataset view.
As an example, we can chain together three stages of filtering labels, matching, and converting to patches:
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F
dataset = foz.load_zoo_dataset("quickstart")
dataset.evaluate_detections("predictions", eval_key="eval")
complex_view = (
dataset
.filter_labels(
"predictions", (
(F("confidence") > 0.3)
& ((F("bounding_box")[2] * F("bounding_box")[3]) > 0.3)
& (F("eval") == "fp")
)
).match(
F("predictions.detections").length() > 2
).to_patches("predictions")
)
print(complex_view)
At the bottom of the print out, we can see the three view stages summarized in order.
Learn more about the ToPatches view stage in the FiftyOne Docs.
Listing available view stages
If you know you want to perform logical operations on your dataset to generate a new view, but aren’t sure exactly how to do so, use the list_view_stages() method to refresh your memory on all of the possibilities.
You can then use the '?' character to print out a view stage’s intended usage as well as some examples.
For example, running dataset.concat? returns the following:
Learn more about available view stages in the FiftyOne Docs.
Add a stage to a view
While every stage is exposed via an associated method on datasets and dataset views, you can also create a stage on its own.
For example, instead of generating a view without predictions using the exclude_fields() method, we can create the corresponding ExcludeFields stage, and then apply this logic to the dataset using the add_stage() method.
import fiftyone as fo
import fiftyone.zoo as foz
# load in dataset
dataset = foz.load_zoo_dataset("quickstart")
# use dataset method directly
view = dataset.exclude_fields(“predictions”)
# alternative: create and add stage
stage = fo.ExcludeFields("predictions")
view = dataset.add_stage(stage)
If we want to apply the same logic to multiple datasets or dataset views, it can be easier and less error-prone to define the stage separately, rather than reproducing identical logic in multiple places.
import fiftyone as fo
import fiftyone.zoo as foz
stage = fo.FilterLabels(
"ground_truth",
F("label").is_in(["cat", "dog"])
)
### apply the cat/dog filter to multiple existing views
cat_dog_view1 = view1.add_stage(stage)
cat_dog_view2 = view2.add_stage(stage)
cat_dog_view3 = view3.add_stage(stage)
Learn more about FilterLabels in the FiftyOne Docs.
Create a stage with MongoDB
If you feel more comfortable working with MongoDB’s query language, you can code up your desired logic in raw MongoDB aggregation pipeline syntax and generate a view stage from this using the Mongo view stage, or the associated mongo() method for datasets and dataset views.
import fiftyone as fo
import fiftyone.zoo as foz
# load in dataset
dataset = foz.load_zoo_dataset("quickstart")
# Stage for getting the second and third samples in the dataset
view = dataset.mongo([{"$skip": 1}, {"$limit": 2}])
## this is equivalent to:
view = dataset.skip(1).limit(2)
This is all possible because FiftyOne uses MongoDB to efficiently store and access datasets.
Dive into the details of FiftyOne’s MongoDB backend in the FiftyOne Docs.
Check for conversions
Some of FiftyOne’s view stages, including ToFrames and ToPatches generate views which differ from the datasets or views to which they are applied not only in content, but also in structure. In the case of ToFrames, the resulting view consisting of images is generated from an initial dataset or view consisting of videos:
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F
dataset = foz.load_zoo_dataset("quickstart-video")
# Create a frames view for an entire video dataset
frames = dataset.to_frames(sample_frames=True)
As another example, applying the SelectGroupSlices stage to a Grouped Dataset results in a view whose media type is not group.
When in question, it is always a good idea to validate the anticipated properties of your generated views by looking at the media_type and other attributes.
Learn more about SelectGroupSlices and Grouped Datasets in the FiftyOne Docs.
Join the FiftyOne community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
- 1,275+ FiftyOne Slack members
- 2,400+ stars on GitHub
- 2,500+ Meetup members
- Used by 222+ repositories
- 54+ contributors
