# fiftyone.core.odm.dataset¶

Documents that track datasets and their sample schemas in the database.

Classes:

 DatasetDocument(*args, **values) Backing document for datasets. SampleFieldDocument(*args, **kwargs) Description of a sample field.

Functions:

 create_field(field_name, ftype[, …]) Creates the fiftyone.core.fields.Field instance defined by the given specification.
fiftyone.core.odm.dataset.create_field(field_name, ftype, embedded_doc_type=None, subfield=None, db_field=None)

Creates the fiftyone.core.fields.Field instance defined by the given specification.

Note

This method is used exclusively to create user-defined (non-default) fields. Any parameters accepted here must be stored on SampleFieldDocument or else datasets will “lose” any additional decorations when they are loaded from the database.

Parameters
Returns

a fiftyone.core.fields.Field instance

class fiftyone.core.odm.dataset.SampleFieldDocument(*args, **kwargs)

Description of a sample field.

Attributes:

 STRICT db_field A unicode string field. embedded_doc_type A unicode string field. ftype A unicode string field. name A unicode string field. subfield A unicode string field.

Methods:

 Hook for doing document level data cleaning before validation is run. clear_field(field_name) Clears the field from the document. Returns a deep copy of the document. fancy_repr([class_name, select_fields, …]) Generates a customizable string representation of the document. field_to_mongo(field_name) field_to_python(field_name, value) from_dict(d[, extended]) Loads the document from a BSON/JSON dictionary. from_field(field) Creates a SampleFieldDocument for a field. Loads the document from a JSON string. get_field(field_name) Gets the field of the document. Get text score from text query has_field(field_name) Determines whether the document has a field of the given name. matches_field(field) Determines whether this sample field matches the given field. set_field(field_name, value[, create]) Sets the value of a field of the document. to_dict([extended]) Serializes this document to a BSON/JSON dictionary. Creates the fiftyone.core.fields.Field specified by this document. to_json([pretty_print]) Serializes the document to a JSON string. to_mongo(*args, **kwargs) Return as SON data ready for use with MongoDB. validate([clean]) Ensure that all fields’ values are valid and that required fields are present.

Classes:

 my_metaclass alias of mongoengine.base.metaclasses.DocumentMetaclass
name

A unicode string field.

ftype

A unicode string field.

subfield

A unicode string field.

embedded_doc_type

A unicode string field.

db_field

A unicode string field.

to_field()

Creates the fiftyone.core.fields.Field specified by this document.

Returns
classmethod from_field(field)

Creates a SampleFieldDocument for a field.

Parameters

field – a fiftyone.core.fields.Field instance

Returns
matches_field(field)

Determines whether this sample field matches the given field.

Parameters

field – a fiftyone.core.fields.Field instance

Returns

True/False

STRICT = False
clean()

Hook for doing document level data cleaning before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)

Clears the field from the document.

Parameters

field_name – the field name

Raises

ValueError – if the field does not exist

copy()

Returns a deep copy of the document.

Returns

a SerializableDocument

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)

Generates a customizable string representation of the document.

Parameters
• class_name (None) – optional class name to use

• select_fields (None) – iterable of field names to restrict to

• exclude_fields (None) – iterable of field names to exclude

• **kwargs – additional key-value pairs to include in the string representation

Returns

a string representation of the document

field_to_mongo(field_name)
field_to_python(field_name, value)
classmethod from_dict(d, extended=False)

Loads the document from a BSON/JSON dictionary.

Parameters
• d – a dictionary

• extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns

a SerializableDocument

classmethod from_json(s)

Loads the document from a JSON string.

Returns

a SerializableDocument

get_field(field_name)

Gets the field of the document.

Parameters

field_name – the field name

Returns

the field value

Raises

AttributeError – if the field does not exist

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

Parameters

field_name – the field name

Returns

True/False

my_metaclass

alias of mongoengine.base.metaclasses.DocumentMetaclass Methods:

 mro() return a type’s method resolution order
set_field(field_name, value, create=False)

Sets the value of a field of the document.

Parameters
• field_name – the field name

• value – the field value

• create (False) – whether to create the field if it does not exist

Raises

ValueError – if field_name is not an allowed field name or does not exist and create == False

to_dict(extended=False)

Serializes this document to a BSON/JSON dictionary.

Parameters

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns

a dict

to_json(pretty_print=False)

Serializes the document to a JSON string.

Parameters

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns

a JSON string

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

validate(clean=True)

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.

class fiftyone.core.odm.dataset.DatasetDocument(*args, **values)

Backing document for datasets.

Miscellaneous:

Attributes:

 STRICT annotation_runs A dictionary field that wraps a standard Python dictionary. brain_methods A dictionary field that wraps a standard Python dictionary. classes A dictionary field that wraps a standard Python dictionary. default_classes A ListField that stores class label strings. default_mask_targets An DictField that stores mapping between integer keys and string targets. evaluations A dictionary field that wraps a standard Python dictionary. frame_fields A field that stores a list of a given type of fiftyone.core.odm.BaseEmbeddedDocument objects. id A field wrapper around MongoDB’s ObjectIds. in_db Whether the document has been inserted into the database. info A dictionary field that wraps a standard Python dictionary. ingest_time The time the document was added to the database, or None if it has not been added to the database. mask_targets A dictionary field that wraps a standard Python dictionary. media_type A unicode string field. name A unicode string field. objects([q_obj]) persistent A boolean field. pk Get the primary key. sample_collection_name A unicode string field. sample_fields A field that stores a list of a given type of fiftyone.core.odm.BaseEmbeddedDocument objects. version A unicode string field.

Methods:

 cascade_save(**kwargs) Recursively save any references and generic references on the document. Hook for doing document level data cleaning before validation is run. clear_field(field_name) Clears the field from the document. Compares the indexes defined in MongoEngine with the ones existing in the database. Returns a deep copy of the document. create_index(keys[, background]) Creates the given indexes if required. delete([signal_kwargs]) Delete the Document from the database. Drops the entire collection associated with this Document type from the database. ensure_index(key_or_list[, background]) Ensure that the given indexes are in place. Checks the document meta data and ensures all the indexes exist. fancy_repr([class_name, select_fields, …]) Generates a customizable string representation of the document. field_to_mongo(field_name) field_to_python(field_name, value) from_dict(d[, extended]) Loads the document from a BSON/JSON dictionary. Loads the document from a JSON string. get_field(field_name) Gets the field of the document. Get text score from text query has_field(field_name) Determines whether the document has a field of the given name. Lists all of the indexes that should be created for given collection. modify([query]) Perform an atomic update of the document in the database and reload the document object using updated version. register_delete_rule(document_cls, …) This method registers the delete rules to apply when removing this object. reload(*fields, **kwargs) Reloads all attributes from the database. save([validate, clean]) Save the Document to the database. select_related([max_depth]) Handles dereferencing of DBRef objects to a maximum depth in order to cut down the number queries to mongodb. set_field(field_name, value[, create]) Sets the value of a field of the document. switch_collection(collection_name[, …]) Temporarily switch the collection for a document instance. switch_db(db_alias[, keep_created]) Temporarily switch the database for a document instance. Returns an instance of DBRef useful in __raw__ queries. to_dict([extended]) Serializes this document to a BSON/JSON dictionary. to_json([pretty_print]) Serializes the document to a JSON string. to_mongo(*args, **kwargs) Return as SON data ready for use with MongoDB. update(**kwargs) Performs an update on the Document A convenience wrapper to update(). validate([clean]) Ensure that all fields’ values are valid and that required fields are present.

Classes:

 my_metaclass alias of mongoengine.base.metaclasses.TopLevelDocumentMetaclass
media_type

A unicode string field.

name

A unicode string field.

sample_collection_name

A unicode string field.

persistent

A boolean field.

info

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

annotation_runs

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

brain_methods

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

evaluations

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

sample_fields

A field that stores a list of a given type of fiftyone.core.odm.BaseEmbeddedDocument objects.

Parameters

document_type – the fiftyone.core.odm.BaseEmbeddedDocument type stored in this field

frame_fields

A field that stores a list of a given type of fiftyone.core.odm.BaseEmbeddedDocument objects.

Parameters

document_type – the fiftyone.core.odm.BaseEmbeddedDocument type stored in this field

classes

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

default_classes

A ListField that stores class label strings.

If this field is not set, its default value is {}.

mask_targets

A dictionary field that wraps a standard Python dictionary.

If this field is not set, its default value is {}.

Parameters

field (None) – an optional Field instance describing the type of the values in the dict

default_mask_targets

An DictField that stores mapping between integer keys and string targets.

If this field is not set, its default value is {}.

version

A unicode string field.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

STRICT = False
cascade_save(**kwargs)

Recursively save any references and generic references on the document.

clean()

Hook for doing document level data cleaning before validation is run.

Any ValidationError raised by this method will not be associated with a particular field; it will have a special-case association with the field defined by NON_FIELD_ERRORS.

clear_field(field_name)

Clears the field from the document.

Parameters

field_name – the field name

Raises

ValueError – if the field does not exist

classmethod compare_indexes()

Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.

copy()

Returns a deep copy of the document.

Returns

a SerializableDocument

classmethod create_index(keys, background=False, **kwargs)

Creates the given indexes if required.

Parameters
• keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering

• background – Allows index creation in the background

delete(signal_kwargs=None, **write_concern)

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parameters
• signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

• write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., w: 2, fsync: True) will wait until at least two servers have recorded the write and will force an fsync on the primary server.

Changed in version 0.10.7: Add signal_kwargs argument

classmethod drop_collection()

Drops the entire collection associated with this Document type from the database.

Raises OperationError if the document has no collection set (i.g. if it is abstract)

Changed in version 0.10.7: OperationError exception raised if no collection available

classmethod ensure_index(key_or_list, background=False, **kwargs)

Ensure that the given indexes are in place. Deprecated in favour of create_index.

Parameters
• key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering

• background – Allows index creation in the background

classmethod ensure_indexes()

Checks the document meta data and ensures all the indexes exist.

Global defaults can be set in the meta - see Defining documents

Note

You can disable automatic index creation by setting auto_create_index to False in the documents meta data

fancy_repr(class_name=None, select_fields=None, exclude_fields=None, **kwargs)

Generates a customizable string representation of the document.

Parameters
• class_name (None) – optional class name to use

• select_fields (None) – iterable of field names to restrict to

• exclude_fields (None) – iterable of field names to exclude

• **kwargs – additional key-value pairs to include in the string representation

Returns

a string representation of the document

field_to_mongo(field_name)
field_to_python(field_name, value)
classmethod from_dict(d, extended=False)

Loads the document from a BSON/JSON dictionary.

Parameters
• d – a dictionary

• extended (False) – whether the input dictionary may contain serialized extended JSON constructs

Returns

a SerializableDocument

classmethod from_json(s)

Loads the document from a JSON string.

Returns

a SerializableDocument

get_field(field_name)

Gets the field of the document.

Parameters

field_name – the field name

Returns

the field value

Raises

AttributeError – if the field does not exist

get_text_score()

Get text score from text query

has_field(field_name)

Determines whether the document has a field of the given name.

Parameters

field_name – the field name

Returns

True/False

id

A field wrapper around MongoDB’s ObjectIds.

property in_db

Whether the document has been inserted into the database.

property ingest_time

The time the document was added to the database, or None if it has not been added to the database.

classmethod list_indexes()

Lists all of the indexes that should be created for given collection. It includes all the indexes from super- and sub-classes.

modify(query=None, **update)

Perform an atomic update of the document in the database and reload the document object using updated version.

Returns True if the document has been updated or False if the document in the database doesn’t match the query.

Note

All unsaved changes that have been made to the document are rejected if the method returns True.

Parameters
• query – the update will be performed only if the document in the database matches the query

• update – Django-style update keyword arguments

my_metaclass

alias of mongoengine.base.metaclasses.TopLevelDocumentMetaclass Methods:

 get_auto_id_names(new_class) Find a name for the automatic ID field for the given new class. mro() return a type’s method resolution order
objects(q_obj=None, **query) = []
property pk

Get the primary key.

classmethod register_delete_rule(document_cls, field_name, rule)

This method registers the delete rules to apply when removing this object.

reload(*fields, **kwargs)

Reloads all attributes from the database.

Parameters
• fields – (optional) args list of fields to reload

• max_depth – (optional) depth of dereferencing to follow

New in version 0.1.2.

Changed in version 0.6: Now chainable

Changed in version 0.9: Can provide specific fields to reload

save(validate=True, clean=True, **kwargs)

Save the Document to the database.

If the document already exists, it will be updated, otherwise it will be created.

Parameters
• validate (True) – validates the document

• clean (True) – call the document’s clean method; requires validate to be True

Returns

self

Handles dereferencing of DBRef objects to a maximum depth in order to cut down the number queries to mongodb.

New in version 0.5.

set_field(field_name, value, create=False)

Sets the value of a field of the document.

Parameters
• field_name – the field name

• value – the field value

• create (False) – whether to create the field if it does not exist

Raises

ValueError – if field_name is not an allowed field name or does not exist and create == False

switch_collection(collection_name, keep_created=True)

Temporarily switch the collection for a document instance.

Only really useful for archiving off data and calling save():

user = User.objects.get(id=user_id)
user.switch_collection('old-users')
user.save()

Parameters
• collection_name (str) – The database alias to use for saving the document

• keep_created (bool) – keep self._created value after switching collection, else is reset to True

Use switch_db if you need to read from another database

switch_db(db_alias, keep_created=True)

Temporarily switch the database for a document instance.

Only really useful for archiving off data and calling save():

user = User.objects.get(id=user_id)
user.switch_db('archive-db')
user.save()

Parameters
• db_alias (str) – The database alias to use for saving the document

• keep_created (bool) – keep self._created value after switching db, else is reset to True

Use switch_collection if you need to read from another collection

to_dbref()

Returns an instance of DBRef useful in __raw__ queries.

to_dict(extended=False)

Serializes this document to a BSON/JSON dictionary.

Parameters

extended (False) – whether to serialize extended JSON constructs such as ObjectIDs, Binary, etc. into JSON format

Returns

a dict

to_json(pretty_print=False)

Serializes the document to a JSON string.

Parameters

pretty_print (False) – whether to render the JSON in human readable format with newlines and indentations

Returns

a JSON string

to_mongo(*args, **kwargs)

Return as SON data ready for use with MongoDB.

update(**kwargs)

Performs an update on the Document A convenience wrapper to update().

Raises OperationError if called on an object that has not yet been saved.

validate(clean=True)

Ensure that all fields’ values are valid and that required fields are present.

Raises ValidationError if any of the fields’ values are found to be invalid.