Dataset Views¶

FiftyOne provides methods that allow you to sort, slice, and search your Dataset using any information that you have added to the Dataset. Performing these actions returns a DatasetView into your Dataset that will that will show only the samples and labels therein that match your criteria.

Overview¶

A DatasetView is returned whenever any sorting, slicing, or searching operation is performed on a Dataset.

You can explicitly create a view that contains an entire dataset via Dataset.view():

import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

view = dataset.view()

print(view)

Dataset:        quickstart
Media type:     image
Num samples:    200
Tags:           ['validation']
Sample fields:
    id:           fiftyone.core.fields.ObjectIdField
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    uniqueness:   fiftyone.core.fields.FloatField
    predictions:  fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
View stages:
    ---

You can access specific information about a view in the natural ways:

len(view)
# 200

view.media_type
# "image"

Note

DatasetView does not hold its contents in-memory; it contains a pipeline of operations that define what samples to will be loaded when the contents of the view are accessed.

Like datasets, you access the samples in a view by iterating over it:

for sample in view:
    # Do something with `sample`

Or, you can access individual samples in a view by their ID or filepath:

sample = view.take(1).first()

print(type(sample))
# fiftyone.core.sample.SampleView

same_sample = view[sample.id]
also_same_sample = view[sample.filepath]

view[other_sample_id]
# KeyError: sample non-existent or not in view

Note

Accessing samples in a DatasetView returns SampleView objects, not Sample objects. The two classes are largely interchangable, but SampleView provides some extra features. See filtering sample contents for more details.

View stages¶

Dataset views encapsulate a pipeline of logical operations that determine which samples appear in the view (and perhaps what subset of their contents).

Each view operation is captured by a ViewStage:

# List available view operations on a dataset
print(dataset.list_view_stages())
# ['exclude', 'exclude_fields', 'exists', ..., 'skip', 'sort_by', 'take']

These operations are conveniently exposed as methods on Dataset instances, in which case they create an initial DatasetView:

# Random set of 100 samples from the dataset
random_view = dataset.take(100)

len(random_view)
# 100

They are also exposed on DatasetView instances, in which case they return another DatasetView with the operation appended to its internal pipeline so that multiple operations can be chained together:

# Sort `random_view` by filepath
sorted_random_view = random_view.sort_by("filepath")

The sections below discuss some interesting view stages in more detail. You can also refer to the fiftyone.core.stages module documentation for examples of using each stage.

Sorting¶

You can use sort_by() to sort the samples in a Dataset or DatasetView by a field of interest. The samples in the returned DatasetView can be sorted in ascending or descending order:

view = dataset.sort_by("filepath")
view = dataset.sort_by("filepath", reverse=True)

You can also sort by expressions!

from fiftyone import ViewField as F

# Sort by number of detections in `Detections` field `ground_truth`
view = dataset.sort_by(F("ground_truth.detections").length(), reverse=True)

print(len(view.first().ground_truth.detections))  # 39
print(len(view.last().ground_truth.detections))  # 0

Slicing¶

You can extract a range of Sample instances from a Dataset using skip() and limit() or, equivalently, by using array slicing:

# Skip the first 2 samples and take the next 3
range_view1 = dataset.skip(2).limit(3)

# Equivalently, using array slicing
range_view2 = dataset[2:5]

Samples can be accessed from views in all the same ways as for datasets. This includes using first() and last() to retrieve the first and last samples in a view, respectively, or accessing a sample directly from a DatasetView by its ID or filepath.

Note

Accessing a sample by its integer index in a DatasetView is not allowed. The best practice is to lookup individual samples by ID or filepath, or use array slicing to extract a range of samples, and iterate over samples in a view.

view[0]
# KeyError: "Accessing samples by numeric index is not supported.
# Use sample IDs, filepaths, or slices"

Shuffling¶

The samples in a Dataset or DatasetView can be randomly shuffled using shuffle():

# Randomly shuffle the order of the samples in the dataset
view1 = dataset.shuffle()

An optional seed can be provided to make the shuffle deterministic:

# Randomly shuffle the samples in the dataset with a fixed seed

view2 = dataset.shuffle(seed=51)
print(view2.first().id)
# 5f31bbfcd0d78c13abe159b1

also_view2 = dataset.shuffle(seed=51)
print(also_view2.first().id)
# 5f31bbfcd0d78c13abe159b1

Random sampling¶

You can extract a random subset of the samples in a Dataset or DatasetView using take():

# Take 5 random samples from the dataset
view1 = dataset.take(5)

An optional seed can be provided to make the sampling deterministic:

# Take 5 random samples from the dataset with a fixed seed

view2 = dataset.take(5, seed=51)
print(view2.first().id)
# 5f31bbfcd0d78c13abe159b1

also_view2 = dataset.take(5, seed=51)
print(also_view2.first().id)
# 5f31bbfcd0d78c13abe159b1

Filtering¶

The real power of DatasetView is the ability to write your own search queries based on your data.

Querying samples¶

You can query for a subset of the samples in a dataset via the match() method. The syntax is:

match_view = dataset.match(expression)

where expression defines the matching expression to use to decide whether to include a sample in the view.

FiftyOne provides powerful ViewField and ViewExpression classes that allow you to use native Python operators to define your match expression. Simply wrap the target field of your sample in a ViewField and then apply comparison, logic, arithmetic or array operations to it to create a ViewExpression. You can use dot notation to refer to fields or subfields of the embedded documents in your samples. Any resulting ViewExpression that returns a boolean is a valid expression!

The code below shows a few examples. See the API reference for ViewExpression for a full list of supported operations.

from fiftyone import ViewField as F

# Populate metadata on all samples
dataset.compute_metadata()

# Samples whose image is less than 48 KB
small_images_view = dataset.match(F("metadata.size_bytes") < 48 * 1024)

# Samples that contain at least one prediction with confidence above 0.99
# or whose label ifs "cat" or "dog"
match = (F("confidence") > 0.99) | (F("label").is_in(("cat", "dog")))
matching_view = dataset.match(
    F("predictions.detections").filter(match).length() > 0
)

Common filters¶

Convenience functions for common queries are also available.

Use the match_tags() method to match samples that have the specified tag(s) in their tags field:

# The validation split of the dataset
val_view = dataset.match_tags("validation")

# Union of the validation and test splits
val_test_view = dataset.match_tags(("validation", "test"))

Use exists() to only include samples for which a given Field exists and is not None:

# The subset of samples where predictions have been computed
predictions_view = dataset.exists("predictions")

Use select() and exclude() to restrict attention to or exclude samples from a view by their IDs:

# Get the IDs of two random samples
sample_ids = [
    dataset.take(1).first().id,
    dataset.take(1).first().id,
]

# Include only samples with the given IDs in the view
selected_view = dataset.select(sample_ids)

# Exclude samples with the given IDs from the view
excluded_view = dataset.exclude(sample_ids)

Filtering sample contents¶

Dataset views can also be used to filter the contents of samples in the view. That’s why DatasetView instances return SampleView objects rather than Sample objects.

SampleView instances represent the content of your samples in all of the usual ways, with some important caveats:

If you modify the contents of a SampleView and then save() it, any changes that you made to the contents of the SampleView will be reflected in the database.
Sample views can exclude fields and filter elements of a field (e.g., omit certain detections from an array of detections in the sample). This means that SampleView instances need not contain all of the information in a sample.
Sample views are not singletons and thus you must explicitly reload() them in order to refresh their contents if the underlying sample has been modified elsewhere. However, extracting a SampleView from a DatasetView always returns the updated version of the sample’s contents.

You can use the select_fields() and exclude_fields() stages to select or exclude fields from the returned SampleView:

for sample in dataset.select_fields("ground_truth"):
    print(sample.id)            # OKAY: `id` is always available
    print(sample.ground_truth)  # OKAY: `ground_truth` was selected
    print(sample.predictions)   # AttributeError: `predictions` was not selected

for sample in dataset.exclude_fields("predictions"):
    print(sample.id)            # OKAY: `id` is always available
    print(sample.ground_truth)  # OKAY: `ground_truth` was not excluded
    print(sample.predictions)   # AttributeError: `predictions` was excluded

The filter_labels() stage is a powerful stage that allows you to filter the contents of Detections, Classifications, Polylines, and Keypoints fields, respectively.

Here are some self-contained examples for each task:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("imagenet-sample")

# Only include samples whose ground truth `label` is "slug" or "conch"
slug_conch_view = dataset.filter_labels(
    "ground_truth", (F("label") == "slug") | (F("label") == "conch")
)

session = fo.launch_app(view=slug_conch_view)

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Bboxes are in [top-left-x, top-left-y, width, height] format
bbox_area = F("bounding_box")[2] * F("bounding_box")[3]

# Only includes predictions whose bounding boxes have an area of at
# least 50% of the image, and only include samples with at least
# one prediction after filtering
large_boxes_view = dataset.filter_labels("predictions", bbox_area >= 0.5)

session = fo.launch_app(view=large_boxes_view)

Note

See the BDD100K dataset in the Dataset Zoo for download instructions.

import fiftyone as fo
import fiftyone.zoo as foz

# The path to the source files that you manually downloaded
source_dir = "/path/to/dir-with-bdd100k-files"

dataset = foz.load_zoo_dataset(
    "bdd100k", split="validation", source_dir=source_dir
)

# Only include polylines that are filled (polygons, not polylines),
# and only include samples with at least one polygon after filtering
polygons_view = dataset.filter_labels("gt_polylines", F("filled") == True)

session = fo.launch_app(view=polygons_view)

Note

This example uses a Keypoint R-CNN model from the Model Zoo.

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Load a keypoint model
model = foz.load_zoo_model("keypoint-rcnn-resnet50-fpn-coco-torch")

# Grab a few samples that have people in them
person_view  = dataset.match(
    F("ground_truth.detections").map(F("label") == "person").length() > 0
).take(4)

person_view.apply_model(model, label_field="rcnn")

# Only include keypoints in the `rcnn_keypoints` field of each
# sample that have at least 10 vertices, and only include samples
# with at least one keypoint instance after filtering
many_points_view = dataset.filter_labels(
    "rcnn_keypoints", F("points").length() >= 10,
)

session = fo.launch_app(view=many_points_view)

You can also use the filter_field() stage to filter the contents of arbitrarily-typed fields:

# Remove tags from samples that don't include the "validation" tag
clean_tags_view = dataset.filter_field("tags", F().contains("validation"))

Note

When you create a DatasetView that contains filtered detections or classifications, the other labels are not removed from the source dataset, even if you save() a SampleView after modifying the filtered detections. This is becauase each label is updated individually, and other labels in the field are left unchanged.

view = dataset.filter_labels("predictions", ...)

for sample in view:
    predictions = sample.predictions

    # Modify the detections in the view
    for detection in predictions.detections:
        detection["new_field"] = True

    # Other detections in the `predictions` field of the samples that
    # did not appear in the `view` are not deleted or modified
    sample.save()

If you do want to delete data from your samples, assign a new value to the field:

view = dataset.filter_labels("predictions", ...)

for sample in view:
    sample.predictions = fo.Detections(...)

    # Existing detections in the `predictions` field of the samples
    # are deleted
    sample.save()

Object patches¶

If your dataset contains label list fields like Detections or Polylines, then you can use to_patches() to create views that contain one sample per object patch in a specified label field of your dataset.

For example, you can extract patches for all ground truth objects in a detection dataset:

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart")

session = fo.launch_app(dataset)

# Convert to ground truth patches
gt_patches = dataset.to_patches("ground_truth")
print(gt_patches)

# View patches in the App
session.view = gt_patches

Dataset:     quickstart
Media type:  image
Num patches: 1232
Tags:        ['validation']
Patch fields:
    id:           fiftyone.core.fields.ObjectIdField
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    sample_id:    fiftyone.core.fields.ObjectIdField
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detection)
View stages:
    1. ToPatches(field='ground_truth')

Or, you could chain view stages to create a view that contains patches for a filtered set of predictions:

# Now extract patches for confident person predictions
person_patches = (
    dataset
    .filter_labels(
        "predictions",
        (F("label") == "person") & (F("confidence") > 0.9)
    )
    .to_patches("predictions")
)
print(person_patches)

# View patches in the App
session.view = person_patches

Note

Did you know? You can convert to object patches view directly from the App!

Object patches views are just like any other dataset view in the sense that:

You can append view stages via the App view bar or views API
Any modifications to label tags that you make via the App’s tagging menu or via API methods like tag_labels() and untag_labels() will be reflected on the source dataset
Any modifications to the Label elements in the patches view that you make by iterating over the contents of the view or calling set_values() will be reflected on the source dataset
Calling save() on an object patches view (typically one that contains additional view stages that filter or modify its contents) will sync any Label edits or deletions with the source dataset

However, because object patches views only contain a subset of the contents of a Sample from the source dataset, there are some differences compared to non-patch views:

Tagging or untagging patches (as opposed to their labels) will not affect the tags of the underlying Sample
Any new fields that you add to an object patches view will not be added to the source dataset

Note

Did you know? You can export object patches as classification datasets!

Evaluation patches¶

If you have run evaluation on predictions from an object detection model, then you can use to_evaluation_patches() to transform the dataset (or a view into it) into a new view that contains one sample for each true positive, false positive, and false negative example.

True positive examples will result in samples with both their ground truth and predicted fields populated, while false positive/negative examples will only have one of their corresponding predicted/ground truth fields populated, respectively.

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Evaluate `predictions` w.r.t. labels in `ground_truth` field
dataset.evaluate_detections(
    "predictions", gt_field="ground_truth", eval_key="eval"
)

session = fo.launch_app(dataset)

# Convert to evaluation patches
eval_patches = dataset.to_evaluation_patches("eval")
print(eval_patches)

# View patches in the App
session.view = eval_patches

Dataset:     quickstart
Media type:  image
Num patches: 5363
Tags:        ['validation']
Patch fields:
    id:           fiftyone.core.fields.ObjectIdField
    filepath:     fiftyone.core.fields.StringField
    tags:         fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:     fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    predictions:  fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    ground_truth: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    sample_id:    fiftyone.core.fields.ObjectIdField
    type:         fiftyone.core.fields.StringField
    iou:          fiftyone.core.fields.FloatField
    crowd:        fiftyone.core.fields.BooleanField
View stages:
    1. ToEvaluationPatches(eval_key='eval')

Note

Did you know? You can convert to evaluation patches view directly from the App!

Note

Refer to the evaluation guide guide for more information about running evaluations and using evaluation patches views to analyze object detection models.

Evaluation patches views are just like any other dataset view in the sense that:

You can append view stages via the App view bar or views API
Any modifications to ground truth or predicted label tags that you make via the App’s tagging menu or via API methods like tag_labels() and untag_labels() will be reflected on the source dataset
Any modifications to the predicted or ground truth Label elements in the patches view that you make by iterating over the contents of the view or calling set_values() will be reflected on the source dataset
Calling save() on an evaluation patches view (typically one that contains additional view stages that filter or modify its contents) will sync any Label edits or deletions with the source dataset

However, because evaluation patches views only contain a subset of the contents of a Sample from the source dataset, there are some differences compared to non-patch views:

Tagging or untagging patches themselves (as opposed to their labels) will not affect the tags of the underlying Sample
Any new fields that you add to an evaluation patches view will not be added to the source dataset

Video views¶

Most view stages naturally support video datasets. For example, stages that refer to fields can be applied to the frame-level fields of video samples by prepending "frames." to the relevent parameters:

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart-video")

# Create a view that only contains vehicles
view = dataset.filter_labels(
    "frames.ground_truth_detections", F("label") == "vehicle"
)

# Compare the number of objects in the view and the source dataset
print(dataset.count("frames.ground_truth_detections.detections"))  # 11345
print(view.count("frames.ground_truth_detections.detections"))  # 7511

In addition, FiftyOne provides a variety of dedicated view stages for performing manipulations that are unique to video data.

Frame views¶

Use to_frames() to create image views into your video datasets that contain one sample per video frame in the dataset.

Note

Did you know? Using to_frames() enables you to execute workflows such as model evaluation and Brain methods that only support image collections to the frames of your video datasets!

In the simplest case, you can create a view that contains a sample for every frame of the videos in a Dataset or DatasetView:

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart-video")

session = fo.launch_app(dataset)

# Create a frames view for the entire dataset
frames = dataset.to_frames()
print(frames)

# Verify that one sample per frame was created
print(dataset.sum("metadata.total_frame_count"))  # 1279
print(len(frames))  # 1279

# View frames in the App
session.view = frames

Dataset:     quickstart-video
Media type:  image
Num samples: 1279
Tags:        []
Sample fields:
    id:                      fiftyone.core.fields.ObjectIdField
    filepath:                fiftyone.core.fields.StringField
    tags:                    fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
    metadata:                fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    sample_id:               fiftyone.core.fields.ObjectIdField
    frame_number:            fiftyone.core.fields.FrameNumberField
    ground_truth_detections: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
View stages:
    1. ToFrames(config=None)

Note

Unless you have configured otherwise, to_frames() will sample the necessary frames from the input video collection into directories of per-frame images when the view is created. For large video datasets, this may take some time and require substantial disk space.

Frames that have previously been sampled will not be resampled, so creating frame views into the same dataset will become faster after the frames have been sampled.

More generally, to_frames() exposes a variety of parameters that you can use to configure the behavior of the video-to-image conversion process.

For example, the snippet below creates a frames view that only contains samples for frames with at least 10 objects, sampling at most one frame per second:

from fiftyone import ViewField as F

#
# Create a frames view that only contains frames with at least 10
# objects, sampled at a maximum frame rate of 1fps
#

num_objects = F("ground_truth_detections.detections").length()
view = dataset.match_frames(num_objects > 10)

frames = view.to_frames(max_fps=1, sparse=True)
print(frames)

# Compare the number of frames in each step
print(dataset.count("frames"))  # 1279
print(view.count("frames"))  # 354
print(len(frames))  # 13

# View frames in the App
session.view = frames

Frame views inherit all frame-level labels from the source video dataset, including their frame number. Each frame sample is also given a sample_id field that records the ID of the parent video sample, and any tags of the parent video sample are also included.

Frame views are just like any other image collection view in the sense that:

You can append view stages via the App view bar or views API
Any modifications to label tags that you make via the App’s tagging menu or via API methods like tag_labels() and untag_labels() will be reflected on the source dataset
Any edits (including additions, modifications, and deletions) to the fields of the samples in a frames view that you make by iterating over the contents of the view or calling set_values() will be reflected on the source dataset
Calling save() on a frames view (typically one that contains additional view stages that filter or modify its contents) will sync any changes to the frames of the underlying video dataset

The only way in which frames views differ from regular image collections is that changes to the tags or metadata fields of frame samples will not be propagated to the frames of the underlying video dataset.

Querying frames¶

You can query for a subset of the frames in a video dataset via match_frames(). The syntax is:

match_view = dataset.match_frames(expression)

where expression defines the matching expression to use to decide whether to include a frame in the view.

FiftyOne provides powerful ViewField and ViewExpression classes that allow you to use native Python operators to define your match expression. Simply wrap the target frame field in a ViewField and then apply comparison, logic, arithmetic or array operations to it to create a ViewExpression. You can use dot notation to refer to fields or subfields of the embedded documents in your frames. Any resulting ViewExpression that returns a boolean is a valid expression!

The snippet below demonstrates a possible workflow. See the API reference for ViewExpression for a full list of supported operations.

import fiftyone as fo
import fiftyone.zoo as foz
from fiftyone import ViewField as F

dataset = foz.load_zoo_dataset("quickstart-video")

# Create a view that only contains frames with at least 10 objects
num_objects = F("ground_truth_detections.detections").length()
view = dataset.match_frames(num_objects > 10)

# Compare the number of frames in each collection
print(dataset.count("frames"))  # 1279
print(view.count("frames"))  # 354

You can also use select_frames() and exclude_frames() to restrict attention to or exclude frames from a view by their IDs:

# Get the IDs of a couple frames
frame_ids = [
    dataset.first().frames.first().id,
    dataset.last().frames.last().id,
]

# Select only the specified frames
selected_view = dataset.select_frames(frame_ids)

# Exclude frames with the given IDs from the view
excluded_view = dataset.exclude_frames(frame_ids)

Visual similarity¶

If you have indexed your dataset by visual similarity, then you can use the sort_by_similarity() stage to programmatically query your data by visual similarity to image(s) or object patch(es) of interest.

Image similarity¶

The example below indexes a dataset by image similarity using compute_similarity() and then uses sort_by_similarity() to sort the dataset by visual similarity to a chosen image:

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Index the dataset by image similarity
fob.compute_similarity(dataset, brain_key="image_sim")

session = fo.launch_app(dataset)

# Select a random query image
query_id = dataset.take(1).first().id

# Sort the samples by visual similarity to the query image
view = dataset.sort_by_similarity(query_id, brain_key="image_sim")
print(view)

# View results in the App
session.view = view

Note

Refer to the Brain guide for more information about generating similarity indexes, and check out the App guide to see how to sort images by visual similarity via point-and-click in the App!

Object similarity¶

The example below indexes the objects in a Detections field of a dataset by similarity using compute_similarity() and then uses sort_by_similarity() to retrieve the 15 most visually similar objects to a chosen object:

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Index the dataset by `ground_truth` object similarity
fob.compute_similarity(
    dataset, patches_field="ground_truth", brain_key="gt_sim"
)

# Convert to ground truth patches view
patches = dataset.to_patches("ground_truth")

# View patches in the App
session = fo.launch_app(view=patches)

# Select a random query object
query_id = patches.take(1).first().id

# Retrieve the 15 most visually similar objects
similar_objects = patches.sort_by_similarity(query_id, k=15, brain_key="gt_sim")

# View results in the App
session.view = similar_objects

Note

Refer to the Brain guide for more information about generating similarity indexes, and check out the App guide to see how to sort objects by visual similarity via point-and-click in the App!

Geolocation¶

If your samples have geolocation data, then you can use the geo_near() and geo_within() stages to filter your data based on their location.

For example, you can use geo_near() to sort your samples by proximity to a location:

import fiftyone as fo
import fiftyone.zoo as foz

TIMES_SQUARE = [-73.9855, 40.7580]

dataset = foz.load_zoo_dataset("quickstart-geo")

# Sort the samples by their proximity to Times Square, and only include
# samples within 5km
view = dataset.geo_near(TIMES_SQUARE, max_distance=5000)

Or, you can use geo_within() to only include samples that lie within a longitude-latitude polygon of your choice:

import fiftyone as fo
import fiftyone.zoo as foz

MANHATTAN = [
    [
        [-73.949701, 40.834487],
        [-73.896611, 40.815076],
        [-73.998083, 40.696534],
        [-74.031751, 40.715273],
        [-73.949701, 40.834487],
    ]
]

dataset = foz.load_zoo_dataset("quickstart-geo")

# Only contains samples in Manhattan
view = dataset.geo_within(MANHATTAN)

Modifying fields¶

In certain situations, you may wish to temporarily modify the values of sample fields in the context of a DatasetView without modifying the underlying dataset. FiftyOne provides the set_field() and map_labels() methods for this purpose.

For example, suppose you would like to rename a group of labels to a single category in order to run your evaluation routine. You can use map_labels() to do this:

ANIMALS = [
    "bear", "bird", "cat", "cow", "dog", "elephant", "giraffe",
    "horse", "sheep", "zebra"
]

# Replace all animal detection's labels with "animal"
mapping = {k: "animal" for k in ANIMALS}
animals_view = dataset.map_labels("predictions", mapping)

counts = animals_view.count_values("predictions.detections.label")
print(counts["animal"])
# 529

Or, suppose you would like to lower bound all confidences of objects in the predictions field of a dataset. You can use set_field() to do this:

# Lower bound all confidences in the `predictions` field to 0.5
bounded_view = dataset.set_field(
    "predictions.detections.confidence",
    F("confidence").max(0.5),
)

print(bounded_view.bounds("predictions.detections.confidence"))
# (0.5, 0.9999035596847534)

The ViewExpression language is quite powerful, allowing you to define complex operations without needing to write an explicit Python loop to perform the desired manipulation.

For example, the snippet below visualizes the top-5 highest confidence predictions for each sample in the quickstart dataset:

from fiftyone import ViewField as F

# Extracts the 5 highest confidence predictions for each sample
top5_preds = F("detections").sort_by("confidence", reverse=True)[:5]

top5_view = (
    dataset
    .set_field("predictions.detections", top5_preds)
    .select_fields("predictions")
)

session = fo.launch_app(view=top5_view)

Saving and cloning¶

Ordinarily, when you define a DatasetView that extracts a specific subset of a dataset and its fields, the underlying Dataset is not modified. However, you can use save() to overwrite the underlying dataset with the contents of a view you’ve created:

from fiftyone import ViewField as F

# Discard all predictions with confidence below 0.3
high_conf_view = dataset.filter_labels("predictions", F("confidence") > 0.3)
high_conf_view.save()

Alternatively, you can create a new Dataset that contains only the contents of a DatasetView using clone():

from fiftyone import ViewField as F

# Create a new dataset that contains only the high confidence predictions
high_conf_view = dataset.filter_labels("predictions", F("confidence") > 0.3)
high_conf_dataset = high_conf_view.clone()

Tips & Tricks¶

Chaining view stages¶

View stages can be chained together to perform complex operations:

from fiftyone import ViewField as F

# Extract the first 5 samples with the "validation" tag, alphabetically by
# filepath, whose images are >= 48 KB
complex_view = (
    dataset
    .match_tags("validation")
    .exists("metadata")
    .match(F("metadata.size_bytes") >= 48 * 1024)  # >= 48 KB
    .sort_by("filepath")
    .limit(5)
)

Filtering detections by area¶

Need to filter your detections by bounding box area? Use this ViewExpression!

from fiftyone import ViewField as F

# Bboxes are in [top-left-x, top-left-y, width, height] format
bbox_area = F("bounding_box")[2] * F("bounding_box")[3]

# Only contains boxes whose area is between 5% and 50% of the image
medium_boxes_view = dataset.filter_labels(
    "predictions", (0.05 <= bbox_area) & (bbox_area < 0.5)
)

FiftyOne stores bounding box coordinates as relative values in [0, 1]. However, you can use the expression below to filter by absolute pixel area:

from fiftyone import ViewField as F

dataset.compute_metadata()

# Computes the area of each bounding box in pixels
bbox_area = (
    F("$metadata.width") * F("bounding_box")[2] *
    F("$metadata.height") * F("bounding_box")[3]
)

# Only contains boxes whose area is between 32^2 and 96^2 pixels
medium_boxes_view = dataset.filter_labels(
    "predictions", (32 ** 2 < bbox_area) & (bbox_area < 96 ** 2)
)

Removing a batch of samples from a dataset¶

You can easily remove a batch of samples from a Dataset by constructing a DatasetView that contains the samples, and then deleting them from the dataset as follows:

# Choose 10 samples at random
unlucky_samples = dataset.take(10)

dataset.delete_samples(unlucky_samples)

Efficiently iterating samples¶

If you have a dataset with larger fields, such as Classifications or Detections, it can be expensive to load entire samples into memory. If, for a particular use case, you are only interested in a subset of fields, you can use Dataset.select_fields() to load only the fields of interest.

Let’s say you have a dataset that looks like this:

Name:           open-images-v4-test
Num samples:    1000
Persistent:     True
Tags:           []
Sample fields:
    id:                       fiftyone.core.fields.ObjectIdField
    filepath:                 fiftyone.core.fields.StringField
    tags:                     fiftyone.core.fields.ListField(StringField)
    metadata:                 fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
    open_images_id:           fiftyone.core.fields.StringField
    groundtruth_image_labels: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Classifications)
    groundtruth_detections:   fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    faster_rcnn:              fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.labels.Detections)
    mAP:                      fiftyone.core.fields.FloatField
    AP_per_class:             fiftyone.core.fields.DictField

and you want to get a list of open_images_id’s for all samples in the dataset. Loading other fields is unnecessary; in fact, using Dataset.select_fields() to load only the open_images_id field speeds up the operation below by ~200X!

import time

start = time.time()
oiids = [s.open_images_id for s in dataset]
print(time.time() - start)
# 38.212332010269165

start = time.time()
oiids = [s.open_images_id for s in dataset.select_fields("open_images_id")]
print(time.time() - start)
# 0.20824909210205078

Dataset Views¶

Overview¶

View stages¶

Sorting¶

Slicing¶

Shuffling¶

Random sampling¶

Filtering¶

Querying samples¶

Common filters¶

Filtering sample contents¶

Object patches¶

Evaluation patches¶

Video views¶

Frame views¶

Querying frames¶

Visual similarity¶

Image similarity¶

Object similarity¶

Geolocation¶

Modifying fields¶

Saving and cloning¶

Tips & Tricks¶

Chaining view stages¶

Filtering detections by area¶

Removing a batch of samples from a dataset¶

Efficiently iterating samples¶

Get FiftyOne for your team