FiftyOne's Video Timeline Catches the One Track a Human Annotator Dropped

A new drawer under the video, not a new panel

Key takeaways

• FiftyOne 1.22.0 adds a video timeline to Explore mode that draws one interval bar for each continuous span a tracked object appears in, so gaps in a track are visible without opening Annotate mode.
• Because Explore mode is read-only, reviewers can audit video annotation quality in the FiftyOne video timeline without accidentally editing labels.
• In the FiftyOne quickstart-video dataset, only 2 of 195 human-annotated tracks (1.0%) contain a gap, and the overall dropout rate is 0.52%.
• The largest track gap in quickstart-video is a vehicle missing from the ground truth for 54 consecutive frames (1.80 seconds) while still on screen. It then picks back up under the same track index.
• A short script that groups frame-level detections by label and track index can find track gaps in any FiftyOne video dataset with tracked detections.

Let's test it on the quickstart-video dataset

What the FiftyOne video timeline actually shows

The measurement: how many tracks have gaps?

Build a track gap check for your own video dataset

tracks = {}
for frame_number, frame in sample.frames.items():
    for det in frame[label_field].detections:
        if det.index is None:
            continue
        tracks.setdefault((det.label, det.index), []).append(frame_number)

# for each track: gap_frames = frames in [first, last] not present
# sort by longest contiguous gap, jump straight to the worst offender
# in the App's video Explore timeline, no Annotate mode required

Where this matters outside a demo dataset

• Autonomous driving annotation QA: vendors delivering per-frame boxes for dashcam or AV log footage are paid for continuous tracks. A 1.8 second dropout in the middle of a clip, on an object that never left the frame, is exactly the kind of defect a spot check misses and a full frame-by-frame review is too slow to catch at scale.
• Traffic and retail camera analytics: dwell-time and counting pipelines that key off a persistent track id will silently double-count or drop an object across a gap like this one, since a system that treats index continuity strictly sees the reappearance as a new object arriving.
• Any outsourced video annotation program: sports analytics, security footage, camera-trap wildlife data, anywhere boxes are drawn by a labeling vendor rather than a model, this same script is the fast way to find the handful of tracks worth escalating before the delivery ships to training.

Try it