Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit.
Wait, what’s FiftyOne?
FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster.
- If you like what you see on GitHub, give the project a star.
- Get started! We’ve made it easy to get up and running in a few minutes.
- Join the FiftyOne Slack community, we’re always happy to help.
Ok, let’s dive into this week’s tips and tricks!
Pre-populating panels in the FiftyOne App
Community Slack member Jiajing asked,
“Is there any way to automatically have a panel open when launching the FiftyOne App?”
Here, Jiajing is referring to Spaces and Panels in the FiftyOne App. As of FiftyOne 0.19, the layout of the FiftyOne App can be thoroughly customized via Spaces, either graphically in the App, or via the SDK. Interactive plotting modules, such as histograms and embeddings are now represented as panels, as well as the sample grid itself. Additionally, you can further customize the appearance of the App by creating your own panel using FiftyOne Plugins!
To configure the FiftyOne App so that a panel appears on launch, you can set attributes of the fiftyone.Space and fiftyone.Panel classes in Python. This is true whether you are working with a built-in panel or a plugin.
If you want the FiftyOne App to open with the samples displayed on top and a histogram of labels displayed below, you can specify this by creating an fo.Space in Python with fo.Panel objects of type Samples and Histograms oriented vertically:
samples_panel = fo.Panel(type="Samples", pinned=True)
histograms_panel = fo.Panel(
type="Histograms",
state=dict(plot="Labels"),
)
spaces = fo.Space(
children=[samples_panel, histograms_panel],
orientation="vertical",
)
And then pass this configuration into fo.launch_app() via the spaces argument:
session = fo.launch_app(dataset, spaces = spaces)
For the Quickstart Dataset, this results in the following display in the FiftyOne App:
Learn more about Spaces and plugins in the FiftyOne Docs.
Accessing a FiftyOne session from a new Python process
Community Slack member Agfian asked,
“Is there a function that returns active sessions in FiftyOne? For instance, I’d like to launch a session from the terminal, and then access that session from a Jupyter notebook. Is this possible?”
Great question, Agfian! Yes, this is absolutely possible. If you launch an App session from one terminal or notebook with session = fo.launch_app(dataset, port=5151), and then in another terminal or notebook you point to the same port, then the session will be the same; specifically, session = fo.launch_app(port=5151) will give you access to the same session.
You can test this out, for instance, by printing the session object in your Python interpreter or notebook cell. You should get all the same information, including identifiers. If you make a change in the App, in both Python processes you should see the changes have propagated.
A few simple ways to try this out are to select a sample in the App and then print session.selected, or to open a histogram panel in the App, and print session.spaces.
Learn more about Sessions and using the FiftyOne App in the FiftyOne Docs.
Filtering samples by patch embeddings
Community Slack member Niki asked,
“Hi all! I am computing the embeddings for object patches and then visualizing these patch embeddings in the FiftyOne App with fob.compute_visualization(dataset, patches_field='predictions', model=model, brain_key='prediction_viz'), but this method results in a Samples panel that is populated with full image samples. Is it possible, after selecting object patches in the Embeddings panel with the lasso tool, to have the images for the object patches displayed in the Samples panel?”
Great question, Niki! The images that display in the Samples panel depend on the DatasetView that is set for the FiftyOne App’s session. If you set the session to be a PatchesView, then the grid will populate with object patches. On the other hand, when you simply launch a session of the App with session = fo.launch_app(dataset) the DatasetView is implicitly set according to the dataset, and the grid is populated with images.
Here is an example of how you could achieve the view you are looking for in the FiftyOne App:
import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart")
model = foz.load_zoo_model("resnet50-imagenet-torch")
patch_view = dataset.to_patches("ground_truth")
patch_view.compute_patch_embeddings(
model,
'ground_truth',
embeddings_field="resnet"
)
fob.compute_visualization(
dataset,
patches_field='ground_truth',
embeddings=resnet,
brain_key='umap',
num_dims=2,
method='umap'
)
session = fo.launch_app(patch_view)
Learn more about PatchesView and working with object patches in the FiftyOne Docs.
Regex on strings in FiftyOne
Community Slack member Daniel asked,
“Hi everyone! I’m trying to retrieve the sample with the filepath that includes a certain string. In my particular case, I want to find the sample with the filepath that includes the string TnFoV_record_08_07_2022_07_40_22/TnFoV_record_08_07_2022_07_40_22-frame0004878.png. Can I do this without looping over all samples in the dataset explicitly?”
This is a great question. In your particular scenario, it looks like you want to check if the filepath ends with the given string, so you can check for the desired condition with the ends_with() method that acts on FiftyOne expressions.
Here is what this looks like in your example:
from fiftyone import ViewField as F
query_str = TnFoV_record_08_07_2022_07_40_22/TnFoV_record_08_07_2022_07_40_22-frame0004878.png
sample = dataset.match(F("filepath").ends_with(query_str)).first()
If your target sample’s filepath does not end with the query string, but still contains the string, you can check for this condition with the contains_str() method. For instance:
import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F
dataset = foz.load_zoo_dataset("quickstart")
## retrieve samples with the string "84" in their filepath
samples84 = dataset.match(F("filepath").contains_str("84"))
Even more generally, FiftyOne supports regex-style pattern matching on StringField objects via the re_match() method.
Learn more about re_match() and filtering with StringField regex in the FiftyOne Docs.
Updating file extensions in FiftyOne
Community Slack member Kevin asked,
“I want to update the extension of the filepath field for the samples of my dataset from PNG to JPG. I’d like to do this without iterating over all samples one by one. Is this possible?”
Hi Kevin. Yes, absolutely! Most of the time, when you want to update a field across all samples in your dataset, there is a more efficient approach than iterating through all samples. Typically, this involves either the set_field() or set_values() method.
In your case, you can use the Values Aggregation to obtain a list of the filepaths for all samples, alter these filepaths in the list, and then use set_values() to set the filepath field with the updated values:
import fiftyone as fo
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart")
## get list of filepaths
fps = dataset.values("filepath")
## change file extensions in list
fps = list(map(lambda x: x.replace("png", "jpg"), fps))
## set filepath field with new values
dataset.set_values('filepath',fps)
Learn more about values(), set_values(), and Aggregations in the FiftyOne Docs.
Join the FiftyOne community!
Join the thousands of engineers and data scientists already using FiftyOne to solve some of the most challenging problems in computer vision today!
- 1,400+ FiftyOne Slack members
- 2,700+ stars on GitHub
- 3,500+ Meetup members
- Used by 259+ repositories
- 58+ contributors
