Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on
Slack,
GitHub, Stack Overflow, and Reddit.
Wait, what’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
[@portabletext/react] Unknown block type "externalImage", specify a component for it in the `components.types` prop
Ok, let’s dive into this week’s tips and tricks!
Pre-populating panels in the FiftyOne App
Community Slack member Jiajing asked,
“Is there any way to automatically have a panel open when launching the FiftyOne App?”
Here, Jiajing is referring to
Spaces and Panels in the FiftyOne App. As of FiftyOne 0.19, the layout of the FiftyOne App can be thoroughly customized via Spaces, either graphically in the App, or via the SDK. Interactive plotting modules, such as
histograms and
embeddings are now represented as panels, as well as the
sample grid itself. Additionally, you can further customize the appearance of the App by creating your own panel using
FiftyOne Plugins!
To configure the FiftyOne App so that a panel appears on launch, you can set attributes of the fiftyone.Space and fiftyone.Panel classes in Python. This is true whether you are working with a built-in panel or a plugin.
If you want the FiftyOne App to open with the samples displayed on top and a histogram of labels displayed below, you can specify this by creating an fo.Space in Python with fo.Panel objects of type Samples and Histograms oriented vertically:
And then pass this configuration into fo.launch_app() via the spaces argument:
For the Quickstart Dataset, this results in the following display in the FiftyOne App:
[@portabletext/react] Unknown block type "externalImage", specify a component for it in the `components.types` prop
Accessing a FiftyOne session from a new Python process
Community Slack member Agfian asked,
“Is there a function that returns active sessions in FiftyOne? For instance, I’d like to launch a session from the terminal, and then access that session from a Jupyter notebook. Is this possible?”
Great question, Agfian! Yes, this is absolutely possible. If you launch an App session from one terminal or notebook with session = fo.launch_app(dataset, port=5151), and then in another terminal or notebook you point to the same port, then the session will be the same; specifically, session = fo.launch_app(port=5151) will give you access to the same session.
You can test this out, for instance, by printing the session object in your Python interpreter or notebook cell. You should get all the same information, including identifiers. If you make a change in the App, in both Python processes you should see the changes have propagated.
A few simple ways to try this out are to select a sample in the App and then print session.selected, or to open a histogram panel in the App, and print session.spaces.
Filtering samples by patch embeddings
Community Slack member Niki asked,
“Hi all! I am computing the embeddings for object patches and then visualizing these patch embeddings in the FiftyOne App with fob.compute_visualization(dataset, patches_field='predictions', model=model, brain_key='prediction_viz'), but this method results in a Samples panel that is populated with full image samples. Is it possible, after selecting object patches in the Embeddings panel with the lasso tool, to have the images for the object patches displayed in the Samples panel?”
Great question, Niki! The images that display in the
Samples panel depend on the
DatasetView that is set for the FiftyOne App’s session. If you set the session to be a
PatchesView, then the grid will populate with object patches. On the other hand, when you simply launch a session of the App with
session = fo.launch_app(dataset) the
DatasetView is implicitly set according to the dataset, and the grid is populated with images.
Here is an example of how you could achieve the view you are looking for in the FiftyOne App:
Regex on strings in FiftyOne
Community Slack member Daniel asked,
“Hi everyone! I'm trying to retrieve the sample with the filepath that includes a certain string. In my particular case, I want to find the sample with the filepath that includes the string TnFoV_record_08_07_2022_07_40_22/TnFoV_record_08_07_2022_07_40_22-frame0004878.png. Can I do this without looping over all samples in the dataset explicitly?”
This is a great question. In your particular scenario, it looks like you want to check if the filepath ends with the given string, so you can check for the desired condition with the ends_with() method that acts on FiftyOne expressions.
Here is what this looks like in your example:
If your target sample’s filepath does not end with the query string, but still contains the string, you can check for this condition with the contains_str() method. For instance:
Even more generally, FiftyOne supports regex-style pattern matching on StringField objects via the re_match() method.
Updating file extensions in FiftyOne
Community Slack member Kevin asked,
“I want to update the extension of the filepath field for the samples of my dataset from PNG to JPG. I’d like to do this without iterating over all samples one by one. Is this possible?”
Hi Kevin. Yes, absolutely! Most of the time, when you want to update a field across all samples in your dataset, there is a more efficient approach than iterating through all samples. Typically, this involves either the set_field() or set_values() method.
In your case, you can use the Values Aggregation to obtain a list of the filepaths for all samples, alter these filepaths in the list, and then use set_values() to set the filepath field with the updated values:
Join the FiftyOne community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
