Using the FiftyOne App¶

The FiftyOne App is a powerful graphical user interface that enables you to visualize, browse, and interact directly with your FiftyOne Datasets.

App environments¶

The FiftyOne App can be used in any environment that you’re working in, from a local IPython shell, to a remote machine or cloud instance, to a Jupyter or Colab notebook. Check out the environments guide for best practices when working in each environment.

Sessions¶

The basic FiftyOne workflow is to open a Python shell and load a Dataset. From there you can launch the FiftyOne App and interact with it programmatically via a session.

Creating a session¶

You can launch an instance of the App by calling launch_app(). This method returns a Session instance, which you can subsequently use to interact programmatically with the App!

import fiftyone as fo

session = fo.launch_app()

App sessions are highly flexible. For example, you can launch launch multiple App instances and connect multiple App instances to the same dataset.

By default, when you’re working in a non-notebook context, the App will be opened in a new tab of your web browser. However, there is also a desktop App that you can install if you would like to run the App as a desktop application.

Note

fo.launch_app() will launch the App asynchronously and return control to your Python process. The App will then remain connected until the process exits.

If you are using the App in a non-interactive script, you should use session.wait() to block execution until you close it manually:

# Launch the App
session = fo.launch_app(...)

# (Perform any additional operations here)

# Blocks execution until the App is closed
session.wait()

Updating a session’s dataset¶

Sessions can be updated to show a new Dataset by updating the Session.dataset property of the session object:

import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("cifar10")

# View the dataset in the App
session.dataset = dataset

Updating a session’s view¶

You can also show a specific view into the current dataset in the App by setting the Session.view property of the session.

For example, the command below loads a DatasetView in the App that shows the first 10 samples in the dataset sorted by their uniqueness field:

session.view = dataset.sort_by("uniqueness").limit(10)

Remote sessions¶

If your data is stored on a remote machine, you can forward a session from the remote machine to your local machine and seemlessly browse your remote dataset from you web browser.

Check out the environments page for more information on possible configurations of local/remote/cloud data and App access.

Remote machine¶

On the remote machine, you can load a Dataset and launch a remote session using either the Python library or the CLI.

Load a Dataset and call launch_app() with the remote=True argument.

# On remote machine

import fiftyone as fo

dataset = fo.load_dataset("<dataset-name>")

session = fo.launch_app(dataset, remote=True)  # optional: port=XXXX

You can use the optional port parameter to choose the port of your remote machine on which to serve the App. The default is 5151, which can also be customized via the default_app_port parameter of your FiftyOne config.

Note that you can manipulate the session object on the remote machine as usual to programmatically interact with the App instance that you’ll connect to locally next.

Run the fiftyone app launch command in a terminal:

# On remote machine

fiftyone app launch <dataset-name> --remote  # optional: --port XXXX

You can use the optional --port flag to choose the port of your remote machine on which to serve the App. The default is 5151, which can also be customized via the default_app_port parameter of your FiftyOne config.

Local machine¶

On the local machine, you can access an App instance connected to the remote session by either manually configuring port forwarding or via the FiftyOne CLI:

Open a new terminal window on your local machine and execute the following command to setup port forwarding to connect to your remote session:

# On local machine
ssh -N -L 5151:127.0.0.1:XXXX [<username>@]<hostname>

Leave this process running and open http://localhost:5151 in your browser to access the App.

In the above, [<username>@]<hostname> specifies the remote machine to connect to, XXXX refers to the port that you chose when you launched the session on your remote machine (the default is 5151), and 5151 specifies the local port to use to connect to the App (and can be customized).

If you have FiftyOne installed on your local machine, you can use the CLI to automatically configure port forwarding and open the App in your browser as follows:

# On local machine
fiftyone app connect --destination [<username>@]<hostname>

If you choose a custom port XXXX on the remote machine, add a --port XXXX flag to the above command.

If you would like to use a custom local port to serve the App, add a --local-port YYYY flag to the above command.

Note

Remote sessions are highly flexible. For example, you can connect to multiple remote sessions and run multiple remote sessions from one machine.

Fields¶

Any labels, tags, and scalar fields can be overlaid on the samples in the App by toggling the corresponding display options on the lefthand side of the App.

Viewing a sample¶

Click a sample to open an expanded view of the sample. This modal also contains information about the fields of the Sample and allows you to access the raw JSON description of the sample.

Filtering sample fields¶

The App provides UI elements in both grid view and expanded sample view that you can use to filter your dataset. To view the available filter options for a field, click the caret icon to the right of the field’s name.

Whenever you modify a filter element, the App will automatically update to show only those samples and/or labels that match the filter.

Note

Did you know? When you have applied filter(s) in the App, a save icon appears in the top-left corner of the sample grid. Clicking this button will convert your filters to an equivalent set of stage(s) in the view bar!

Using the view bar¶

The view bar makes all of the powerful searching, sorting, and filtering operations provided by dataset views available directly in the App.

Note

Any changes to the current view that you make in the view bar are automatically reflected in the DatasetView exposed by the Session.view property of the App’s session object.

Statistics tabs¶

The Labels, Scalars, and Tags tabs in the App let you visualize different statistics about your dataset.

Note

The statistics in these tabs automatically update to reflect the current view that you have loaded in the App, or the entire dataset if no view is loaded.

The Labels tab shows distributions of the label values for each labels field that you’ve added to your dataset. For example, you may have histograms of ground truth labels and one more sets of model predictions.

The Scalars tab shows distributions for numeric (integer or float) or categorical (e.g., string) primitive fields that you’ve added to your dataset. For example, if you computed uniqueness on your dataset, a histogram of uniqueness values will be displayed under the Scalars tab.

The Tags tab shows the distribution of any tags that you’ve added to your dataset.

Selecting samples¶

As previously explained, the Session object created when you launch the App lets you interact with the App from your Python process.

One common workflow is to select samples visually in the App and then access the data for the selected samples in Python. To perform this workflow, first select some samples in the App:

The selected samples checkmark in the options row in the upper-left corner of the sample grid records the number of samples that you have currently selected. You can also take actions such as updating the view to only show (or exclude) the currently selected samples.

Once sample Tagging also automatically applies to selected samples or their labels when any samples are selected. See tagging for more details.

You can also access the Session.selected property of your session to retrieve the IDs of the currently selected samples in the App:

# Print the IDs of the currently selected samples
print(session.selected)

# Create a view containing only the selected samples
selected_view = dataset.select(session.selected)

['5ef0eef405059ebb0ddfa6cc',
 '5ef0eef405059ebb0ddfa7c4',
 '5ef0eef405059ebb0ddfa86e',
 '5ef0eef405059ebb0ddfa93c']

Selecting labels¶

You can also use the App to select individual labels within samples. You can use this functionality to visually show/hide labels of interest in the App; or you can access the data for the selected labels from Python, for example by creating a DatasetView that includes/excludes the selected labels.

To perform this workflow, open the expanded sample modal by clicking on a sample in the App. Then click on individual labels to select them:

Selected labels will appear with dotted lines around them. The example above shows selecting an object detection, but classifications, polygons, polylines, segmentations, and keypoints can be selected as well.

When you have selected labels in the App, you can use the selected labels options in the upper-right (the orange checkmark button) to hide these labels from view or exclude all other labels.

You can also access the Session.selected_labels property of your session to retrieve information about the currently selected labels in the App:

# Print information about the currently selected samples in the App
fo.pprint(session.selected_labels)

# Create a view containing only the selected labels
selected_view = dataset.select_labels(session.selected_labels)

# Create a view containing everything except the selected labels
excluded_view = dataset.exclude_labels(session.selected_labels)

[
    {
        'object_id': '5f99d2eb36208058abbfc02a',
        'sample_id': '5f99d2eb36208058abbfc030',
        'field': 'ground_truth',
    },
    {
        'object_id': '5f99d2eb36208058abbfc02b',
        'sample_id': '5f99d2eb36208058abbfc030',
        'field': 'ground_truth',
    },
    ...
]

Tags and tagging¶

Tagging is a first-class citizen in FiftyOne, as both Sample and Label instances have a tags attribute that you can use to store arbitrary string tags for your data.

The FiftyOne API provides methods like tag_samples() and tag_labels() that you can use to programmatically manage the tags on your dataset. However, the App also provides a convenient UI for interactively adding, removing, and filtering by Sample and Label tags.

You can tag or untag batches of samples/labels in the App by clicking on the tag icon above the sample grid.

For example, take the following steps to tag all labels in the predictions field of a dataset:

Make sure that predictions is the only Label field checked in the filters sidebar
Click the tag icon in the top-left corner of the grid
Select Labels, type in the tag, and then click Apply

You can also use the tag menu to remove existing tags.

Note

Any tagging operations that you perform using the tagging UI above the sample grid will be applied to your current view, respecting any filters or show/hide checkboxes you have applied in the filters sidebar, unless you have selected individual samples, in which case the operation will only apply to the selected samples.

The App also supports tagging data in individual samples when you have opened the expanded sample view by clicking on a sample. The tag icon is located in the top-right corner of the modal.

Note

Any tagging operations that you perform using the tagging UI in expanded sample mode will be applied to the current sample, respecting any filters or show/hide checkboxes you have applied, unless you have selected individual labels, in which case the operation will only apply to the selected labels. The latter may span multiple samples.

If your dataset has sample or label tags, you can use the SAMPLE TAGS and LABEL TAGS sections of the filters sidebar to filter by your tags.

When you click the eye icon next to a sample tag, your view will update to only include samples with the tag(s) you have selected. When you click the eye icon next to a label tag, your view will update to only include labels with tag(s) you have selected, and any samples with no matches will be automatically excluded.

Note

Did you know? When you have applied filter(s) in the App, a save icon appears in the top-left corner of the sample grid. Clicking this button will convert your filters to an equivalent set of stage(s) in the view bar!

Configuring the App¶

The behavior of the App can be configured in various ways. The code sample below shows the basic pattern for customizing the App on a one-off basis:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Create a custom App config
app_config = fo.AppConfig()
app_config.show_confidence = False
app_config.show_attributes = True

session = fo.launch_app(dataset, config=app_config)

You can also reconfigure a live Session by editing its session.config property and calling session.refresh() to apply the changes:

# Customize the config of a live Session
session.config.show_confidence = True
session.config.show_attributes = True

# Refresh the session to apply the changes
session.refresh()

See this page for more information about configuring the App.