Using the FiftyOne App¶

The FiftyOne App is a powerful graphical user interface that enables you to visualize, browse, and interact directly with your FiftyOne Datasets.

App environments¶

The FiftyOne App can be used in any environment that you’re working in, from a local IPython shell, to a remote machine or cloud instance, to a Jupyter or Colab notebook. Check out the environments guide for best practices when working in each environment.

Sessions¶

The basic FiftyOne workflow is to open a Python shell and load a Dataset. From there you can launch the FiftyOne App and interact with it programmatically via a session.

Creating a session¶

You can launch an instance of the App by calling launch_app(). This method returns a Session instance, which you can subsequently use to interact programmatically with the App!

import fiftyone as fo

session = fo.launch_app()

App sessions are highly flexible. For example, you can launch launch multiple App instances and connect multiple App instances to the same dataset.

By default, when you’re working in a non-notebook context, the App will be opened in a new tab of your web browser. However, there is also a desktop App that you can install if you would like to run the App as a desktop application.

Note

fo.launch_app() will launch the App asynchronously and return control to your Python process. The App will then remain open until you close it or the process exits.

If you are using the App in a non-interactive script, you should use session.wait() to block execution until you close it manually:

# Launch the App
session = fo.launch_app(...)

# (Perform any additional operations here)

# Blocks execution until the App is closed
session.wait()

Updating a session’s dataset¶

Sessions can be updated to show a new Dataset by updating the Session.dataset property of the session object:

import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("cifar10")

# View the dataset in the App
session.dataset = dataset

Updating a session’s view¶

You can also show a specific view into the current dataset in the App by setting the Session.view property of the session.

For example, the command below loads a DatasetView in the App that shows the first 10 samples in the dataset sorted by their uniqueness field:

session.view = dataset.sort_by("uniqueness").limit(10)

Remote sessions¶

If your data is stored on a remote machine, you can forward a session from the remote machine to your local machine and seemlessly browse your remote dataset via the App.

Check out the environments page for more information on possible configurations of local/remote/cloud data and App access.

Remote machine¶

On the remote machine, you can load a Dataset and launch a remote session using either the Python library or the CLI.

Load a Dataset and call launch_app() with the remote=True argument.

# On remote machine

import fiftyone as fo

dataset = fo.load_dataset("<dataset-name>")

session = fo.launch_app(dataset, remote=True)  # optional: port=XXXX

You can use the optional port parameter to choose the port of your remote machine on which to serve the App. The default is 5151, which can be customized via the default_app_port parameter of your FiftyOne config.

Note that you can manipulate the session object on the remote machine as usual to programmatically interact with the App instance that you’ll connect to next.

Run the fiftyone app launch command in a terminal:

# On remote machine

fiftyone app launch <dataset-name> --remote  # optional: --port XXXX

You can use the optional --port flag to choose the port of your remote machine on which to serve the App. The default is 5151, which can be customized via the default_app_port parameter of your FiftyOne config.

Local machine¶

On the local machine, you can launch an App instance connected to a remote session using either the Python library or the CLI (recommended).

Open two terminal windows on the local machine.

The first step is to configure port forwarding of the remote machine’s port to your local machine. Do this by running the following command in one terminal and leave the process running:

# On local machine

ssh -N -L 5151:127.0.0.1:5151 username@remote_machine_ip

If you chose a custom port XXXX on the remote machine, substitute it for the second 5151 in the above command.

If you would like to use a custom local port to serve the App, substitute it for the first 5151 in the above command.

In the other terminal, launch the FiftyOne App locally by starting Python and running the following commands:

# On local machine

import fiftyone as fo

fo.launch_app()  # optional: port=YYYY

If you chose a custom local port when configuring port forwarding, specify it via the port parameter of launch_app().

On the local machine, use the fiftyone app connect command to connect to a remote session:

# On local machine

fiftyone app connect --destination username@remote_machine_ip

If you choose a custom port XXXX on the remote machine, add a --port XXXX flag to the above command.

If you would like to use a custom local port to serve the App, add a --local-port YYYY flag to the above command.

Note

Remote sessions are highly flexible. For example, you can connect to multiple remote sessions and run multiple remote sessions from one machine.

Fields¶

Any labels, tags, and scalar fields can be overlaid on the samples in the App by toggling the corresponding display options on the lefthand side of the App.

Viewing a sample¶

Double-click a sample to open an expanded view of the sample. This modal also contains information about the fields of the Sample and allows you to access the raw JSON description of the sample.

Filtering sample fields¶

The App provides UI elements in both grid view and expanded sample view that you can use to filter your dataset. To view the available filter options for a field, click the caret icon to the right of the field’s name.

Whenever you modify a filter element, the App will automatically update to show only those samples and/or labels that match the filter.

Using the view bar¶

The view bar makes all of the powerful searching, sorting, and filtering operations provided by dataset views available directly in the App.

Note

Any changes to the current view that you make in the view bar are automatically reflected in the DatasetView exposed by the Session.view property of the App’s session object.

Statistics tabs¶

The Labels, Scalars, and Tags tabs in the App let you visualize different statistics about your dataset.

Note

The statistics in these tabs automatically update to reflect the current view that you have loaded in the App, or the entire dataset if no view is loaded.

The Labels tab shows distributions of the label values for each labels field that you’ve added to your dataset. For example, you may have histograms of ground truth labels and one more sets of model predictions.

The Scalars tab shows distributions for numeric (integer or float) or categorical (e.g., string) primitive fields that you’ve added to your dataset. For example, if you computed uniqueness on your dataset, a histogram of uniqueness values will be displayed under the Scalars tab.

The Tags tab shows the distribution of any tags that you’ve added to your dataset.

Selecting samples¶

As previously explained, the Session object created when you launch the App lets you interact with the App from your Python process.

One common workflow is to select samples visually in the App and then access the data for the selected samples in Python. To perform this workflow, first select some samples in the App:

The selected samples dropdown on the upper-left of the sample grid records the number of samples that you have currently selected. You can also take actions such as updating the view to only show (or exclude) the currently selected samples.

You can also access the Session.selected property of your session to retrieve the IDs of the currently selected samples in the App:

# Print the IDs of the currently selected samples
print(session.selected)

# Create a view containing only the selected samples
selected_view = dataset.select(session.selected)

['5ef0eef405059ebb0ddfa6cc',
 '5ef0eef405059ebb0ddfa7c4',
 '5ef0eef405059ebb0ddfa86e',
 '5ef0eef405059ebb0ddfa93c']

Selecting objects¶

You can also use the App to select individual objects within samples. You can use this functionality to visually show/hide objects of interest in the App; or you can access the data for the selected objects from Python, for example by creating a DatasetView that includes/excludes the selected objects.

To perform this workflow, open the expanded sample modal by double-clicking on a sample in the App. Then click on individual objects to select them:

Selected objects will appear with dotted lines around them. The example above shows selecting an object detection, but polygons, polylines, segmentations, and keypoints can be selected as well.

When you have selected objects in the App, you can use the selected objects dropdown menu under Fields to take actions such as hiding the selected samples from view.

You can also access the Session.selected_objects property of your session to retrieve information about the currently selected objects in the App:

# Print information about the currently selected samples in the App
fo.pprint(session.selected_objects)

# Create a view containing only the selected objects
selected_view = dataset.select_objects(session.selected_objects)

# Create a view containing everything except the selected objects
excluded_view = dataset.exclude_objects(session.selected_objects)

[
    {
        'object_id': '5f99d2eb36208058abbfc02a',
        'sample_id': '5f99d2eb36208058abbfc030',
        'field': 'ground_truth',
    },
    {
        'object_id': '5f99d2eb36208058abbfc02b',
        'sample_id': '5f99d2eb36208058abbfc030',
        'field': 'ground_truth',
    },
    ...
]

Configuring the App¶

The behavior of the App can be configured in various ways. The code sample below shows the basic pattern for customizing the App on a one-off basis:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Create a custom App config
app_config = fo.AppConfig()
app_config.show_confidence = False
app_config.show_attributes = False

session = fo.launch_app(dataset, config=app_config)

You can even reconfigure a live Session by editing its session.config property and calling session.refresh() to apply the changes:

# Customize the config of a live Session
session.config.show_confidence = True
session.config.show_attributes = True

# Refresh the session to apply the changes
session.refresh()

See this page for more information about configuring the App.