Wait, what’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
- If you like what you see on GitHub, give the project a star.
- Get started! We’ve made it easy to get up and running in a few minutes.
- Join the FiftyOne Slack community, we’re always happy to help.
Ok, let’s dive into this week’s tips and tricks!
Pre-populating panels in the FiftyOne App
Community Slack member Jiajing asked,
“Is there any way to automatically have a panel open when launching the FiftyOne App?”
Here, Jiajing is referring to Spaces and Panels in the FiftyOne App. As of FiftyOne 0.19, the layout of the FiftyOne App can be thoroughly customized via Spaces, either graphically in the App, or via the SDK. Interactive plotting modules, such as histograms and embeddings are now represented as panels, as well as the sample grid itself. Additionally, you can further customize the appearance of the App by creating your own panel using FiftyOne Plugins!
To configure the FiftyOne App so that a panel appears on launch, you can set attributes of the
fiftyone.Panel classes in Python. This is true whether you are working with a built-in panel or a plugin.
If you want the FiftyOne App to open with the samples displayed on top and a histogram of labels displayed below, you can specify this by creating an
fo.Space in Python with
fo.Panel objects of type
Histograms oriented vertically:
samples_panel = fo.Panel(type="Samples", pinned=True) histograms_panel = fo.Panel( type="Histograms", state=dict(plot="Labels"), ) spaces = fo.Space( children=[samples_panel, histograms_panel], orientation="vertical", )
And then pass this configuration into
fo.launch_app() via the
session = fo.launch_app(dataset, spaces = spaces)
For the Quickstart Dataset, this results in the following display in the FiftyOne App:
Accessing a FiftyOne session from a new Python process
Community Slack member Agfian asked,
“Is there a function that returns active sessions in FiftyOne? For instance, I’d like to launch a session from the terminal, and then access that session from a Jupyter notebook. Is this possible?”
Great question, Agfian! Yes, this is absolutely possible. If you launch an App session from one terminal or notebook with
session = fo.launch_app(dataset, port=5151), and then in another terminal or notebook you point to the same port, then the session will be the same; specifically,
session = fo.launch_app(port=5151) will give you access to the same session.
You can test this out, for instance, by printing the
session object in your Python interpreter or notebook cell. You should get all the same information, including identifiers. If you make a change in the App, in both Python processes you should see the changes have propagated.
A few simple ways to try this out are to select a sample in the App and then print
session.selected, or to open a histogram panel in the App, and print
Filtering samples by patch embeddings
Community Slack member Niki asked,
“Hi all! I am computing the embeddings for object patches and then visualizing these patch embeddings in the FiftyOne App with
fob.compute_visualization(dataset, patches_field='predictions', model=model, brain_key='prediction_viz'), but this method results in a Samples panel that is populated with full image samples. Is it possible, after selecting object patches in the Embeddings panel with the lasso tool, to have the images for the object patches displayed in the Samples panel?”
Great question, Niki! The images that display in the Samples panel depend on the
DatasetView that is set for the FiftyOne App’s session. If you set the session to be a
PatchesView, then the grid will populate with object patches. On the other hand, when you simply launch a session of the App with
session = fo.launch_app(dataset) the
DatasetView is implicitly set according to the dataset, and the grid is populated with images.
Here is an example of how you could achieve the view you are looking for in the FiftyOne App:
import fiftyone as fo import fiftyone.brain as fob import fiftyone.zoo as foz dataset = foz.load_zoo_dataset("quickstart") model = foz.load_zoo_model("resnet50-imagenet-torch") patch_view = dataset.to_patches("ground_truth") patch_view.compute_patch_embeddings( model, 'ground_truth', embeddings_field="resnet" ) fob.compute_visualization( dataset, patches_field='ground_truth', embeddings=resnet, brain_key='umap', num_dims=2, method='umap' ) session = fo.launch_app(patch_view)
Regex on strings in FiftyOne
Community Slack member Daniel asked,
“Hi everyone! I’m trying to retrieve the sample with the filepath that includes a certain string. In my particular case, I want to find the sample with the filepath that includes the string
TnFoV_record_08_07_2022_07_40_22/TnFoV_record_08_07_2022_07_40_22-frame0004878.png. Can I do this without looping over all samples in the dataset explicitly?”
This is a great question. In your particular scenario, it looks like you want to check if the filepath ends with the given string, so you can check for the desired condition with the
ends_with() method that acts on FiftyOne expressions.
Here is what this looks like in your example:
from fiftyone import ViewField as F query_str = TnFoV_record_08_07_2022_07_40_22/TnFoV_record_08_07_2022_07_40_22-frame0004878.png sample = dataset.match(F("filepath").ends_with(query_str)).first()
If your target sample’s filepath does not end with the query string, but still contains the string, you can check for this condition with the
contains_str() method. For instance:
import fiftyone as fo import fiftyone.zoo as foz from fiftyone import ViewField as F dataset = foz.load_zoo_dataset("quickstart") ## retrieve samples with the string "84" in their filepath samples84 = dataset.match(F("filepath").contains_str("84"))
Even more generally, FiftyOne supports regex-style pattern matching on
StringField objects via the
Updating file extensions in FiftyOne
Community Slack member Kevin asked,
“I want to update the extension of the
filepath field for the samples of my dataset from PNG to JPG. I’d like to do this without iterating over all samples one by one. Is this possible?”
Hi Kevin. Yes, absolutely! Most of the time, when you want to update a field across all samples in your dataset, there is a more efficient approach than iterating through all samples. Typically, this involves either the
In your case, you can use the
Values Aggregation to obtain a list of the filepaths for all samples, alter these filepaths in the list, and then use
set_values() to set the
filepath field with the updated values:
import fiftyone as fo import fiftyone.zoo as foz dataset = foz.load_zoo_dataset("quickstart") ## get list of filepaths fps = dataset.values("filepath") ## change file extensions in list fps = list(map(lambda x: x.replace("png", "jpg"), fps)) ## set filepath field with new values dataset.set_values('filepath',fps)
Join the FiftyOne community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
- 1,400+ FiftyOne Slack members
- 2,700+ stars on GitHub
- 3,500+ Meetup members
- Used by 259+ repositories
- 58+ contributors