Table of Contents
Contents

fiftyone.core.odm.database

Database utilities.

Copyright 2017-2021, Voxel51, Inc.
voxel51.com

Functions:

aggregate(collection, pipelines)

Executes one or more aggregations on a collection.

bulk_write(ops, coll[, ordered])

Performs a batch of write operations on a collection.

delete_annotation_run(name, anno_key[, dry_run])

Deletes the annotation run with the given key from the dataset with the given name.

delete_annotation_runs(name[, dry_run])

Deletes all annotation runs from the dataset with the given name.

delete_brain_run(name, brain_key[, dry_run])

Deletes the brain method run with the given key from the dataset with the given name.

delete_brain_runs(name[, dry_run])

Deletes all brain method runs from the dataset with the given name.

delete_dataset(name[, dry_run])

Deletes the dataset with the given name.

delete_evaluation(name, eval_key[, dry_run])

Deletes the evaluation run with the given key from the dataset with the given name.

delete_evaluations(name[, dry_run])

Deletes all evaluations from the dataset with the given name.

drop_database()

Drops the database.

drop_orphan_collections([dry_run])

Drops all orphan collections from the database.

drop_orphan_run_results([dry_run])

Drops all orphan run results from the database.

establish_db_conn(config)

Establishes the database connection.

export_collection(docs, json_path[, key, …])

Exports the collection to disk in JSON format.

export_document(doc, json_path)

Exports the document to disk in JSON format.

get_async_db_conn()

Returns an async connection to the database.

get_collection_stats(collection_name)

Sets stats about the collection.

get_db_client()

Returns a database client.

get_db_conn()

Returns a connection to the database.

import_collection(json_path)

Imports the collection from JSON on disk.

import_document(json_path)

Imports a document from JSON on disk.

insert_documents(docs, coll[, ordered])

Inserts a list of documents into a collection.

list_collections()

Returns a list of all collection names in the database.

list_datasets()

Returns the list of available FiftyOne datasets.

stream_collection(collection_name)

Streams the contents of the collection to stdout.

sync_database()

Syncs all pending database writes to disk.
fiftyone.core.odm.database.establish_db_conn(config)

Establishes the database connection.

If fiftyone.config.database_uri is defined, then we connect to that URI. Otherwise, a fiftyone.core.service.DatabaseService is created.

Parameters

config – a fiftyone.core.config.FiftyOneConfig

Raises

  • ConnectionError – if a connection to mongod could not be established

  • FiftyOneConfigError – if fiftyone.config.database_uri is not defined and mongod could not be found

  • ServiceExecutableNotFound – if fiftyone.core.service.DatabaseService startup was attempted, but mongod was not found in fiftyone.db.bin

  • RuntimeError – if the mongod found does not meet FiftyOne’s requirements, or validation could not occur

fiftyone.core.odm.database.aggregate(collection, pipelines)

Executes one or more aggregations on a collection.

Multiple aggregations are executed using multiple threads, and their results are returned as lists rather than cursors.

Parameters

  • collection – a pymongo.collection.Collection or motor.motor_tornado.MotorCollection

  • pipelines – a MongoDB aggregation pipeline or a list of pipelines

Returns

  • If a single pipeline is provided, a pymongo.command_cursor.CommandCursor or motor.motor_tornado.MotorCommandCursor is returned

  • If multiple pipelines are provided, each cursor is extracted into a list and the list of lists is returned

fiftyone.core.odm.database.get_db_client()

Returns a database client.

Returns

a pymongo.mongo_client.MongoClient

fiftyone.core.odm.database.get_db_conn()

Returns a connection to the database.

Returns

a pymongo.database.Database

fiftyone.core.odm.database.get_async_db_conn()

Returns an async connection to the database.

Returns

a motor.motor_tornado.MotorDatabase

fiftyone.core.odm.database.drop_database()

Drops the database.

fiftyone.core.odm.database.sync_database()

Syncs all pending database writes to disk.

fiftyone.core.odm.database.list_collections()

Returns a list of all collection names in the database.

Returns

a list of all collection names

fiftyone.core.odm.database.drop_orphan_collections(dry_run=False)

Drops all orphan collections from the database.

Orphan collections are collections that are not associated with any known dataset or other collections used by FiftyOne.

Parameters

dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.drop_orphan_run_results(dry_run=False)

Drops all orphan run results from the database.

Orphan run results are results that are not associated with any known dataset.

Parameters

dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.stream_collection(collection_name)

Streams the contents of the collection to stdout.

Parameters

collection_name – the name of the collection

fiftyone.core.odm.database.get_collection_stats(collection_name)

Sets stats about the collection.

Parameters

collection_name – the name of the collection

Returns

a stats dict

fiftyone.core.odm.database.export_document(doc, json_path)

Exports the document to disk in JSON format.

Parameters

  • doc – a BSON document dict

  • json_path – the path to write the JSON file

fiftyone.core.odm.database.export_collection(docs, json_path, key='documents', num_docs=None)

Exports the collection to disk in JSON format.

Parameters

  • docs – an iteraable containing the documents to export

  • json_path – the path to write the JSON file

  • key ("documents") – the field name under which to store the documents

  • num_docs (None) – the total number of documents. If omitted, this must be computable via len(docs)

fiftyone.core.odm.database.import_document(json_path)

Imports a document from JSON on disk.

Parameters

json_path – the path to the document

Returns

a BSON document dict

fiftyone.core.odm.database.import_collection(json_path)

Imports the collection from JSON on disk.

Parameters

json_path – the path to the collection on disk

Returns

a BSON dict

fiftyone.core.odm.database.insert_documents(docs, coll, ordered=False)

Inserts a list of documents into a collection.

The _id field of the input documents will be populated if it is not already set.

Parameters

  • docs – the list of BSON document dicts to insert

  • coll – a pymongo collection instance

  • ordered (False) – whether the documents must be inserted in order

fiftyone.core.odm.database.bulk_write(ops, coll, ordered=False)

Performs a batch of write operations on a collection.

Parameters

  • ops – a list of pymongo operations

  • coll – a pymongo collection instance

  • ordered (False) – whether the operations must be performed in order

fiftyone.core.odm.database.list_datasets()

Returns the list of available FiftyOne datasets.

This is a low-level implementation of dataset listing that does not call fiftyone.core.dataset.list_datasets(), which is helpful if a database may be corrupted.

Returns

a list of Dataset names

fiftyone.core.odm.database.delete_dataset(name, dry_run=False)

Deletes the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Parameters

  • name – the name of the dataset

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_annotation_run(name, anno_key, dry_run=False)

Deletes the annotation run with the given key from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_annotation_run(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • anno_key – the annotation key

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_annotation_runs(name, dry_run=False)

Deletes all annotation runs from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_annotation_runs(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_brain_run(name, brain_key, dry_run=False)

Deletes the brain method run with the given key from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_brain_run(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • brain_key – the brain key

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_brain_runs(name, dry_run=False)

Deletes all brain method runs from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_brain_runs(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_evaluation(name, eval_key, dry_run=False)

Deletes the evaluation run with the given key from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_evaluation(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • eval_key – the evaluation key

  • dry_run (False) – whether to log the actions that would be taken but not perform them

fiftyone.core.odm.database.delete_evaluations(name, dry_run=False)

Deletes all evaluations from the dataset with the given name.

This is a low-level implementation of deletion that does not call fiftyone.core.dataset.load_dataset() or fiftyone.core.collections.SampleCollection.delete_evaluations(), which is helpful if a dataset’s backing document or collections are corrupted and cannot be loaded via the normal pathways.

Note that, as this method does not load fiftyone.core.runs.Run instances, it does not call fiftyone.core.runs.Run.cleanup().

Parameters

  • name – the name of the dataset

  • dry_run (False) – whether to log the actions that would be taken but not perform them