fiftyone.utils.eval.utils

Evaluation utilities.

Copyright 2017-2022, Voxel51, Inc.

voxel51.com

Functions:

compute_ious(preds, gts[, iscrowd, …])	Computes the pairwise IoUs between the predicted and ground truth objects.
make_iscrowd_fcn(iscrowd_attr)	Returns a boolean function that determines whether a fiftyone.core.labels.Label is a crowd by checking for an attribute with the given name.

fiftyone.utils.eval.utils.make_iscrowd_fcn(iscrowd_attr)

Returns a boolean function that determines whether a fiftyone.core.labels.Label is a crowd by checking for an attribute with the given name.

Parameters

iscrowd_attr – the name of the crowd attribute

Returns

a boolean function

fiftyone.utils.eval.utils.compute_ious(preds, gts, iscrowd=None, classwise=False, use_masks=False, use_boxes=False, tolerance=None, error_level=1)

Computes the pairwise IoUs between the predicted and ground truth objects.

Parameters

• preds – a list of predicted fiftyone.core.labels.Detection or fiftyone.core.labels.Polyline instances

• gt_field – a list of ground truth fiftyone.core.labels.Detection or fiftyone.core.labels.Polyline instances

• iscrowd (None) – an optional name of a boolean attribute or boolean function to apply to each label that determines whether a ground truth object is a crowd. If provided, the area of the predicted object is used as the “union” area for IoU calculations involving crowd objects

• classwise (False) – whether to consider objects with different label values as always non-overlapping (True) or to compute IoUs for all objects regardless of label (False)

• use_masks (False) – whether to compute IoUs using the instances masks in the mask attribute of the provided objects, which must be fiftyone.core.labels.Detection instances

• use_boxes (False) – whether to compute IoUs using the bounding boxes of the provided fiftyone.core.labels.Polyline instances rather than using their actual geometries

• tolerance (None) – a tolerance, in pixels, when generating approximate polylines for instance masks. Typical values are 1-3 pixels

• error_level (1) –

  the error level to use when manipulating instance masks or polylines. Valid values are:

  • 0: raise geometric errors that are encountered

  • 1: log warnings if geometric errors are encountered

  • 2: ignore geometric errors

  If error_level > 0, any calculation that raises a geometric error will default to an IoU of 0

Returns

a num_preds x num_gts array of IoUs