fiftyone.core.odm.sample¶

Backing document classes for fiftyone.core.sample.Sample instances.

Class hierarchy:

SerializableDocument
├── NoDatasetSampleDocument
└── DatasetSampleDocument
├── my_custom_dataset
├── another_dataset
└── ...


Implementation details

When a new fiftyone.core.sample.Sample is created, its _doc attribute is an instance of NoDatasetSampleDocument:

import fiftyone as fo

sample = fo.Sample()
sample._doc  # NoDatasetSampleDocument


When a new fiftyone.core.dataset.Dataset is created, its _sample_doc_cls attribute holds a dynamically created subclass of DatasetSampleDocument whose name is the name of the dataset’s sample collection:

dataset = fo.Dataset(name="my_dataset")
dataset._sample_doc_cls  # my_dataset(DatasetSampleDocument)


When a sample is added to a dataset, its _doc attribute is changed from type NoDatasetSampleDocument to type dataset._sample_doc_cls:

dataset.add_sample(sample)
sample._doc  # my_dataset(DatasetSampleDocument)


Classes:

 DatasetSampleDocument(*args, **values) Base class for sample documents backing samples in datasets. NoDatasetSampleDocument(**kwargs) Backing document for samples that have not been added to a dataset.
class fiftyone.core.odm.sample.DatasetSampleDocument(*args, **values)

Base class for sample documents backing samples in datasets.

All fiftyone.core.dataset.Dataset._sample_doc_cls classes inherit from this class.

Attributes:

 STRICT collection_name field_names filepath A unicode string field. id An Object ID field. in_db Whether the document has been inserted into the database. media_type metadata A field that stores instances of a given type of fiftyone.core.odm.BaseEmbeddedDocument object. pk Get the primary key. tags A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

Methods:

 add_field(path, ftype[, embedded_doc_type, …]) Adds a new field or embedded field to the document, if necessary. add_implied_field(path, value[, …]) Adds the field or embedded field to the document, if necessary, inferring the field type from the provided value. cascade_save(**kwargs) Recursively save any references and generic references on the document. Hook for doing document level data cleaning (usually validation or assignment) before validation is run. clear_field(field_name) Compares the indexes defined in MongoEngine with the ones existing in the database. Returns a deep copy of the document. create_index(keys[, background]) Creates the given indexes if required. delete([signal_kwargs]) Delete the Document from the database. Drops the entire collection associated with this Document type from the database. ensure_index(key_or_list[, background]) Ensure that the given indexes are in place. Checks the document meta data and ensures all the indexes exist. fancy_repr([class_name, select_fields, …]) Generates a customizable string representation of the document. field_to_mongo(field_name) field_to_python(field_name, value) from_dict(d[, extended]) Loads the document from a BSON/JSON dictionary. Loads the document from a JSON string. get_field(field_name) get_field_schema([ftype, embedded_doc_type, …]) Returns a schema dictionary describing the fields of this document. Get text score from text query has_field(field_name) Returns an iterator over the (name, value) pairs of the public fields of the document. Lists all indexes that should be created for the Document collection. merge(doc[, merge_lists, merge_dicts, overwrite]) Merges the contents of the given document into this document. merge_field_schema(schema[, expand_schema, …]) Merges the field schema into this document. modify([query]) Perform an atomic update of the document in the database and reload the document object using updated version. register_delete_rule(document_cls, …) This method registers the delete rules to apply when removing this object. reload(*fields, **kwargs) Reloads all attributes from the database. save([validate, clean, safe]) Saves the document to the database. select_related([max_depth]) Handles dereferencing of DBRef objects to a maximum depth in order to cut down the number queries to mongodb. set_field(field_name, value[, create, …]) switch_collection(collection_name[, …]) Temporarily switch the collection for a document instance. switch_db(db_alias[, keep_created]) Temporarily switch the database for a document instance. Returns an instance of DBRef useful in __raw__ queries. to_dict([extended]) Serializes this document to a BSON/JSON dictionary. to_json([pretty_print]) Serializes the document to a JSON string. to_mongo(*args, **kwargs) Return as SON data ready for use with MongoDB. update(**kwargs) Performs an update on the Document A convenience wrapper to update(). validate([clean]) Ensure that all fields’ values are valid and that required fields are present.

Classes:

 my_metaclass alias of mongoengine.base.metaclasses.TopLevelDocumentMetaclass
id

An Object ID field.

Parameters
• description (None) – an optional description

• info (None) – an optional info dict

filepath

A unicode string field.

Parameters
• description (None) – an optional description

• info (None) – an optional info dict

tags

A list field that wraps a standard Field, allowing multiple instances of the field to be stored as a list in the database.

If this field is not set, its default value is [].

Parameters
• field (None) – an optional Field instance describing the type of the list elements

• description (None) – an optional description

• info (None) – an optional info dict

metadata

A field that stores instances of a given type of fiftyone.core.odm.BaseEmbeddedDocument object.

Parameters
• document_type – the fiftyone.core.odm.BaseEmbeddedDocument type stored in this field

• description (None) – an optional description

• info (None) – an optional info dict

property media_type
STRICT = False
classmethod add_field(path, ftype, embedded_doc_type=None, subfield=None, fields=None, description=None, info=None, expand_schema=True, recursive=True, validate=True, **kwargs)

Adds a new field or embedded field to the document, if necessary.

Parameters
Returns

True/False whether one or more fields or embedded fields were added to the document or its children

Raises

ValueError – if a field in the schema is not compliant with an existing field of the same name

classmethod add_implied_field(path, value, expand_schema=True, dynamic=False, recursive=True, validate=True)

Adds the field or embedded field to the document, if necessary, inferring the field type from the provided value.

Parameters
• path – the field name or embedded.field.name

• value – the field value

• expand_schema (True) – whether to add new fields to the schema (True) or simply validate that the field already exists with a consistent type (False)

• dynamic (False) – whether to declare dynamic embedded document fields

• recursive (True) – whether to recursively add embedded document fields

• validate (True) – whether to validate the field against an existing field at the same path

Returns

True/False whether one or more fields or embedded fields were added to the document or its children

Raises

ValueError – if a field in the schema is not compliant with an existing field of the same name

cascade_save(**kwargs)

Recursively save any references and generic references on the document.

clean()

Hook for doing document level data cleaning (usually validation or assignment) before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)
property collection_name
classmethod compare_indexes()

Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.

copy()

Returns a deep copy of the document.

Returns

a SerializableDocument

classmethod create_index(keys, background=False, **kwargs)

Creates the given indexes if required.

Parameters
• keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering

• background – Allows index creation in the background

delete(signal_kwargs=None, **write_concern)

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parameters
• signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

• write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., w: 2, fsync: True) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

classmethod drop_collection()

Drops the entire collection associated with this Document type from the database.

Raises OperationError if the document has no collection set (i.g. if it is abstract)

classmethod ensure_index(key_or_list, background=False, **kwargs)

Ensure that the given indexes are in place. Deprecated in favour of create_index.

Parameters
• key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering

• background – Allows index creation in the background

classmethod ensure_indexes()

Checks the document meta data and ensures all the indexes exist.

Global defaults can be set in the meta - see Defining documents

By default, this will get called automatically upon first interaction with the Document collection (query, save, etc) so unless you disabled auto_create_index, you shouldn’t have to call this manually.

Note

You can disable automatic index creation by setting auto_create_index to False in the documents meta data

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)

Generates a customizable string representation of the document.

Parameters
• class_name (None) – optional class name to use

• select_fields (None) – iterable of field names to restrict to

• exclude_fields (None) – iterable of field names to exclude

• **kwargs – additional key-value pairs to include in the string representation

Returns

a string representation of the document

property field_names
field_to_mongo(field_name)
field_to_python(field_name, value)
classmethod from_dict(d, extended=False)

Loads the document from a BSON/JSON dictionary.

Parameters
• d – a dictionary

• extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns

a SerializableDocument

classmethod from_json(s)

Loads the document from a JSON string.

Returns

a SerializableDocument

get_field(field_name)
classmethod get_field_schema(ftype=None, embedded_doc_type=None, include_private=False)

Returns a schema dictionary describing the fields of this document.

If the document belongs to a dataset, the schema will apply to all documents in the collection.

Parameters
• ftype (None) – an optional field type to which to restrict the returned schema. Must be a subclass of fiftyone.core.fields.Field

• embedded_doc_type (None) – an optional embedded document type to which to restrict the returned schema. Must be a subclass of fiftyone.core.odm.BaseEmbeddedDocument

• include_private (False) – whether to include fields that start with _ in the returned schema

Returns

a dictionary mapping field names to field types

get_text_score()

Get text score from text query

has_field(field_name)
property in_db

Whether the document has been inserted into the database.

iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns

an iterator that emits (name, value) tuples

classmethod list_indexes()

Lists all indexes that should be created for the Document collection. It includes all the indexes from super- and sub-classes.

Note that it will only return the indexes’ fields, not the indexes’ options

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)

Merges the contents of the given document into this document.

Parameters
• doc – a SerializableDocument of same type as this document

• merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

• merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

• overwrite (True) – whether to overwrite (True) or skip (False) existing fields

classmethod merge_field_schema(schema, expand_schema=True, recursive=True, validate=True)

Merges the field schema into this document.

Parameters
• schema – a dictionary mapping field names or embedded.field.namesto fiftyone.core.fields.Field instances

• expand_schema (True) – whether to add new fields to the schema (True) or simply validate that fields already exist with consistent types (False)

• recursive (True) – whether to recursively merge embedded document fields

• validate (True) – whether to validate the field against an existing field at the same path

Returns

True/False whether any new fields were added

Raises

ValueError – if a field in the schema is not compliant with an existing field of the same name or a new field is found but expand_schema == False

modify(query=None, **update)

Perform an atomic update of the document in the database and reload the document object using updated version.

Returns True if the document has been updated or False if the document in the database doesn’t match the query.

Note

All unsaved changes that have been made to the document are rejected if the method returns True.

Parameters
• query – the update will be performed only if the document in the database matches the query

• update – Django-style update keyword arguments

my_metaclass

alias of mongoengine.base.metaclasses.TopLevelDocumentMetaclass Methods:

 get_auto_id_names(new_class) Find a name for the automatic ID field for the given new class. mro Return a type’s method resolution order.
property pk

Get the primary key.

classmethod register_delete_rule(document_cls, field_name, rule)

This method registers the delete rules to apply when removing this object.

reload(*fields, **kwargs)

Reloads all attributes from the database.

Parameters
• fields – (optional) args list of fields to reload

• max_depth – (optional) depth of dereferencing to follow

save(validate=True, clean=True, safe=False, **kwargs)

Saves the document to the database.

If the document already exists, it will be updated, otherwise it will be created.

Parameters
• validate (True) – whether to validate the document

• clean (True) – whether to call the document’s clean() method. Only applicable when validate is True

• safe (False) – whether to reload() the document before raising any validation errors

Returns

self

Handles dereferencing of DBRef objects to a maximum depth in order to cut down the number queries to mongodb.

set_field(field_name, value, create=True, validate=True, dynamic=False)
switch_collection(collection_name, keep_created=True)

Temporarily switch the collection for a document instance.

Only really useful for archiving off data and calling save():

user = User.objects.get(id=user_id)
user.switch_collection('old-users')
user.save()

Parameters
• collection_name (str) – The database alias to use for saving the document

• keep_created (bool) – keep self._created value after switching collection, else is reset to True

Use switch_db if you need to read from another database

switch_db(db_alias, keep_created=True)

Temporarily switch the database for a document instance.

Only really useful for archiving off data and calling save():

user = User.objects.get(id=user_id)
user.switch_db('archive-db')
user.save()

Parameters
• db_alias (str) – The database alias to use for saving the document

• keep_created (bool) – keep self._created value after switching db, else is reset to True

Use switch_collection if you need to read from another collection

to_dbref()

Returns an instance of DBRef useful in __raw__ queries.

to_dict(extended=False)

Serializes this document to a BSON/JSON dictionary.

Parameters

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns

a dict

to_json(pretty_print=False)

Serializes the document to a JSON string.

Parameters

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns

a JSON string

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

update(**kwargs)

Performs an update on the Document A convenience wrapper to update().

Raises OperationError if called on an object that has not yet been saved.

validate(clean=True)

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.odm.sample.NoDatasetSampleDocument(**kwargs)

Backing document for samples that have not been added to a dataset.

Methods:

 clear_field(field_name) Returns a deep copy of the document. fancy_repr([class_name, select_fields, …]) Generates a customizable string representation of the document. from_dict(d[, extended]) Loads the document from a BSON/JSON dictionary. Loads the document from a JSON string. get_field(field_name) has_field(field_name) Returns an iterator over the (name, value) pairs of the public fields of the document. merge(doc[, merge_lists, merge_dicts, overwrite]) Merges the contents of the given document into this document. set_field(field_name, value[, create, …]) to_dict([extended]) to_json([pretty_print]) Serializes the document to a JSON string.

Attributes:

default_fields = {'_media_type': <fiftyone.core.fields.StringField object>, '_rand': <fiftyone.core.fields.FloatField object>, 'filepath': <fiftyone.core.fields.StringField object>, 'id': <fiftyone.core.fields.ObjectIdField object>, 'metadata': <fiftyone.core.fields.EmbeddedDocumentField object>, 'tags': <fiftyone.core.fields.ListField object>}
default_fields_ordered = ('id', 'filepath', 'tags', 'metadata', '_media_type', '_rand')
property media_type
clear_field(field_name)
copy()

Returns a deep copy of the document.

Returns

a SerializableDocument

delete()
fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)

Generates a customizable string representation of the document.

Parameters
• class_name (None) – optional class name to use

• select_fields (None) – iterable of field names to restrict to

• exclude_fields (None) – iterable of field names to exclude

• **kwargs – additional key-value pairs to include in the string representation

Returns

a string representation of the document

property field_names
classmethod from_dict(d, extended=False)

Loads the document from a BSON/JSON dictionary.

Parameters
• d – a dictionary

• extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns

a SerializableDocument

classmethod from_json(s)

Loads the document from a JSON string.

Returns

a SerializableDocument

get_field(field_name)
has_field(field_name)
property in_db
iter_fields()

Returns an iterator over the (name, value) pairs of the public fields of the document.

Returns

an iterator that emits (name, value) tuples

merge(doc, merge_lists=True, merge_dicts=True, overwrite=True)

Merges the contents of the given document into this document.

Parameters
• doc – a SerializableDocument of same type as this document

• merge_lists (True) – whether to merge the elements of top-level list fields rather than treating the list as a single value

• merge_dicts (True) – whether to recursively merge the contents of top-level dict fields rather than treating the dict as a single value

• overwrite (True) – whether to overwrite (True) or skip (False) existing fields

reload()
save()
set_field(field_name, value, create=True, validate=True, dynamic=False)
to_dict(extended=False)
to_json(pretty_print=False)

Serializes the document to a JSON string.

Parameters

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns

a JSON string