fiftyone.zoo¶

Module contents¶

The FiftyOne Dataset Zoo.

This package defines a collection of open source datasets made available for download via FiftyOne.

Copyright 2017-2020, Voxel51, Inc.
voxel51.com

Classes

`ZooDataset`	Base class for datasets made available in the FiftyOne Dataset Zoo.
`ZooDatasetInfo`(zoo_dataset, dataset_type, …)	Class containing info about a dataset in the FiftyOne Dataset Zoo.
`ZooDatasetSplitInfo`(split, num_samples)	Class containing info about a split of a dataset in the FiftyOne Dataset Zoo.

Functions

`download_zoo_dataset`(name[, split, splits, …])	Downloads the dataset of the given name from the FiftyOne Dataset Zoo.
`find_zoo_dataset`(name[, split])	Returns the directory containing the given zoo dataset.
`get_zoo_dataset`(name)	Returns the `ZooDataset` instance for the dataset with the given name.
`list_downloaded_zoo_datasets`([base_dir])	Returns information about the zoo datasets that have been downloaded.
`list_zoo_datasets`()	Returns the list of available datasets in the FiftyOne Dataset Zoo.
`load_zoo_dataset`(name[, split, splits, …])	Loads the dataset of the given name from the FiftyOne Dataset Zoo as a `fiftyone.core.dataset.Dataset`.
`load_zoo_dataset_info`(name[, dataset_dir])	Loads the `ZooDatasetInfo` for the specified zoo dataset.

fiftyone.zoo.list_zoo_datasets()¶

Returns the list of available datasets in the FiftyOne Dataset Zoo.

Returns: a list of dataset names

fiftyone.zoo.list_downloaded_zoo_datasets(base_dir=None)¶

Returns information about the zoo datasets that have been downloaded.

Parameters: base_dir (None) – the base directory to search. By default, fo.config.default_dataset_dir is used
Returns: a dict mapping dataset names to (dataset dir, ZooDatasetInfo) tuples

fiftyone.zoo.download_zoo_dataset(name, split=None, splits=None, dataset_dir=None)¶

Downloads the dataset of the given name from the FiftyOne Dataset Zoo.

Any dataset splits that already exist in the specified directory are not re-downloaded.

Parameters

name – the name of the zoo dataset to download. Call list_zoo_datasets() to see the available datasets
split (None) – ("train", "validation", "test"). If neither split nor splits are provided, all available splits are downloaded. Consult the documentation for the ZooDataset you specified to see the supported splits
splits (None) – a list of splits to download, if applicable. Typical values are ("train", "validation", "test"). If neither split nor splits are provided, all available splits are downloaded. Consult the documentation for the ZooDataset you specified to see the supported splits
dataset_dir (None) – the directory into which to download the dataset. By default, fiftyone.core.dataset.get_default_dataset_dir() is used

Returns

tuple of

info: the fiftyone.zoo.ZooDatasetInfo for the dataset
dataset_dir: the directory containing the dataset

fiftyone.zoo.load_zoo_dataset(name, split=None, splits=None, dataset_name=None, dataset_dir=None, download_if_necessary=True, drop_existing_dataset=False, **kwargs)¶

Loads the dataset of the given name from the FiftyOne Dataset Zoo as a fiftyone.core.dataset.Dataset.

By default, the dataset will be downloaded if it does not already exist in the specified directory.

If you do not specify a custom dataset_name and you have previously loaded the same zoo dataset and split(s) into FiftyOne, the existing fiftyone.core.dataset.Dataset will be returned.

Parameters

name – the name of the zoo dataset to load. Call list_zoo_datasets() to see the available datasets
split (None) – ("train", "validation", "test"). If neither split nor splits are provided, all available splits are loaded. Consult the documentation for the ZooDataset you specified to see the supported splits
splits (None) – a list of splits to load, if applicable. Typical values are ("train", "validation", "test"). If neither split nor splits are provided, all available splits are loaded. Consult the documentation for the ZooDataset you specified to see the supported splits
dataset_name (None) – an optional name to give the returned fiftyone.core.dataset.Dataset. By default, a name will be constructed based on the dataset and split(s) you are loading
dataset_dir (None) – the directory in which the dataset is stored or will be downloaded. By default, fiftyone.core.dataset.get_default_dataset_dir() is used
download_if_necessary (True) – whether to download the dataset if it is not found in the specified dataset directory
drop_existing_dataset (False) – whether to drop an existing dataset with the same name if it exists
**kwargs – optional keyword arguments to pass to the constructor of the fiftyone.utils.data.importers.DatasetImporter for the dataset via the syntax DatasetImporter(dataset_dir, **kwargs)

Returns

a fiftyone.core.dataset.Dataset

fiftyone.zoo.find_zoo_dataset(name, split=None)¶

Returns the directory containing the given zoo dataset.

If a split is provided, the path to the dataset split is returned; otherwise, the path to the root directory is returned.

The dataset must be downloaded. Use download_zoo_dataset() to download datasets.

Parameters

name – the name of the zoo dataset
split (None) –

Returns

the directory containing the dataset

fiftyone.zoo.load_zoo_dataset_info(name, dataset_dir=None)¶

Loads the ZooDatasetInfo for the specified zoo dataset.

The dataset must be downloaded. Use download_zoo_dataset() to download datasets.

Parameters

name – the name of the zoo dataset
dataset_dir (None) – the directory in which the dataset is stored. By default, fiftyone.core.dataset.get_default_dataset_dir() is used

Returns

the ZooDatasetInfo for the dataset

Raises

OSError – if the dataset has not been downloaded

fiftyone.zoo.get_zoo_dataset(name)¶

Returns the ZooDataset instance for the dataset with the given name.

If the dataset is available from multiple sources, the default source is used.

Parameters: name – the name of the zoo dataset
Returns: the ZooDataset instance

class fiftyone.zoo.ZooDatasetInfo(zoo_dataset, dataset_type, num_samples, downloaded_splits=None, classes=None)¶

Bases: eta.core.serial.Serializable

Class containing info about a dataset in the FiftyOne Dataset Zoo.

Parameters

zoo_dataset – the ZooDataset instance for the dataset
dataset_type – the fiftyone.types.dataset_types.Dataset type of the dataset
num_samples – the total number of samples in all downloaded splits of the dataset
downloaded_splits (None) – a dict of ZooDatasetSplitInfo instances describing the downloaded splits of the dataset, if applicable
classes (None) – a list of class label strings

property name¶: The name of the dataset.

property zoo_dataset¶: The fully-qualified class string for the ZooDataset of the dataset.

property dataset_type¶: The fully-qualified class string of the fiftyone.types.dataset_types.Dataset type.

property supported_splits¶: A tuple of supported splits for the dataset, or None if the dataset does not have splits.

get_zoo_dataset()¶

Returns the ZooDataset instance for the dataset.

Returns: a ZooDataset instance

get_dataset_type()¶

Returns the fiftyone.types.dataset_types.Dataset type instance for the dataset.

Returns: a fiftyone.types.dataset_types.Dataset instance

is_split_downloaded(split)¶

Whether the given dataset split is downloaded.

Parameters: split – the dataset split
Returns: True/False

attributes()¶

Returns a list of class attributes to be serialized.

Returns: a list of class attributes

classmethod from_dict(d)¶

Loads a ZooDatasetInfo from a JSON dictionary.

Parameters: d – a JSON dictionary
Returns: a ZooDatasetInfo

copy()¶

Returns a deep copy of the object.

Returns: a Serializable instance

custom_attributes(dynamic=False, private=False)¶

Returns a customizable list of class attributes.

By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).

Parameters

dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False

Returns

a list of class attributes

classmethod from_json(path, *args, **kwargs)¶

Constructs a Serializable object from a JSON file.

Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.

Parameters

path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()

Returns

an instance of the Serializable class

classmethod from_str(s, *args, **kwargs)¶

Constructs a Serializable object from a JSON string.

Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.

Parameters

s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()

Returns

an instance of the Serializable class

classmethod get_class_name()¶: Returns the fully-qualified class name string of this object.

serialize(reflective=False)¶

Serializes the object into a dictionary.

Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.

Parameters: reflective – whether to include reflective attributes when serializing the object. By default, this is False
Returns: a JSON dictionary representation of the object

to_str(pretty_print=True, **kwargs)¶

Returns a string representation of this object.

Parameters

pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()

Returns

a string representation of the object

write_json(path, pretty_print=False, **kwargs)¶

Serializes the object and writes it to disk.

Parameters

path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()

class fiftyone.zoo.ZooDatasetSplitInfo(split, num_samples)¶

Bases: eta.core.serial.Serializable

Class containing info about a split of a dataset in the FiftyOne Dataset Zoo.

Parameters

split – the name of the split
num_samples – the number of samples in the split

attributes()¶

Returns a list of class attributes to be serialized.

Returns: a list of class attributes

classmethod from_dict(d)¶

Loads a ZooDatasetSplitInfo from a JSON dictionary.

Parameters: d – a JSON dictionary
Returns: a ZooDatasetSplitInfo

copy()¶

Returns a deep copy of the object.

Returns: a Serializable instance

custom_attributes(dynamic=False, private=False)¶

Returns a customizable list of class attributes.

By default, all attributes in vars(self) are returned, minus private attributes (those starting with “_”).

Parameters

dynamic – whether to include dynamic properties, e.g., those defined by getter/setter methods or the @property decorator. By default, this is False
private – whether to include private properties, i.e., those starting with “_”. By default, this is False

Returns

a list of class attributes

classmethod from_json(path, *args, **kwargs)¶

Constructs a Serializable object from a JSON file.

Subclasses may override this method, but, by default, this method simply reads the JSON and calls from_dict(), which subclasses must implement.

Parameters

path – the path to the JSON file on disk
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()

Returns

an instance of the Serializable class

classmethod from_str(s, *args, **kwargs)¶

Constructs a Serializable object from a JSON string.

Subclasses may override this method, but, by default, this method simply parses the string and calls from_dict(), which subclasses must implement.

Parameters

s – a JSON string representation of a Serializable object
*args – optional positional arguments for self.from_dict()
**kwargs – optional keyword arguments for self.from_dict()

Returns

an instance of the Serializable class

classmethod get_class_name()¶: Returns the fully-qualified class name string of this object.

serialize(reflective=False)¶

Serializes the object into a dictionary.

Serialization is applied recursively to all attributes in the object, including element-wise serialization of lists and dictionary values.

Parameters: reflective – whether to include reflective attributes when serializing the object. By default, this is False
Returns: a JSON dictionary representation of the object

to_str(pretty_print=True, **kwargs)¶

Returns a string representation of this object.

Parameters

pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is True
**kwargs – optional keyword arguments for self.serialize()

Returns

a string representation of the object

write_json(path, pretty_print=False, **kwargs)¶

Serializes the object and writes it to disk.

Parameters

path – the output path
pretty_print – whether to render the JSON in human readable format with newlines and indentations. By default, this is False
**kwargs – optional keyword arguments for self.serialize()

class fiftyone.zoo.ZooDataset¶

Bases: object

Base class for datasets made available in the FiftyOne Dataset Zoo.

property name¶: The name of the dataset.

property supported_splits¶: A tuple of supported splits for the dataset, or None if the dataset does not have splits.

property has_splits¶: Whether the dataset has splits.

has_split(split)¶

Whether the dataset has the given split.

Parameters: split – the dataset split
Returns: True/False

static load_info(dataset_dir)¶

Loads the ZooDatasetInfo from the given dataset directory.

Parameters: dataset_dir – the directory in which to construct the dataset
Returns: the ZooDatasetInfo for the dataset

static get_split_dir(dataset_dir, split)¶

Returns the directory for the given split of the dataset.

Parameters

dataset_dir – the dataset directory
split – the dataset split

Returns

the directory that will/does hold the specified split

static get_info_path(dataset_dir)¶

Returns the path to the ZooDatasetInfo for the dataset.

Parameters: dataset_dir – the dataset directory
Returns: the path to the ZooDatasetInfo

download_and_prepare(dataset_dir, split=None, splits=None)¶

Downloads the dataset and prepares it for use in the given directory.

If the requested splits have already been downloaded, they are not re-downloaded.

Parameters

dataset_dir – the directory in which to construct the dataset
split (None) – split nor splits are provided, the full dataset is downloaded
splits (None) – a list of splits to download, if applicable. If neither split nor splits are provided, the full dataset is downloaded

Returns

the ZooDatasetInfo for the dataset