Pinpoint Annotation Issues with In-Depth Metrics and Spatiotemporal Comments in Voxel51 Annotation
Author
Founding Machine Learning Engineer
Degrees in Physics, Computer Science, and Machine Learning, but I primarily spend my time as an ML engineer focused on Computer Vision. Recent interests have ranged from object detection, to video understanding, to data visualization.
See all articles by Eric Hofesmann
In this article
Try Voxel51 for yourself
When an annotation project falls behind or rejections climb in the review queue, sample counts alone don't reveal where the process is struggling. Teams need to understand how much work each annotator completes, how long annotating samples takes, why labels are getting rejected, and how often submissions pass review. Without that visibility, it’s harder to identify performance patterns across annotators, workflow stages, and datasets.
Voxel51 Annotation’s latest release introduces annotation metrics to give teams a more detailed view of annotation activity and review outcomes across their workflows. The release also adds spatiotemporal comments, allowing reviewers to attach feedback to specific locations in images and frames in video.
Key takeaways
- Voxel51 adds annotation metrics that track throughput, time spent, label activity, and review outcomes.
- Voxel51 annotation metrics can cover one workflow, all workflows in a dataset, or every dataset across a deployment.
- Voxel51 annotation metrics break down activity by annotator and by workflow stage, with separate measures for the Annotate and Review stages.
- Spatiotemporal comments in Voxel51 let reviewers attach feedback to specific locations in images and specific frames in video and multimodal datasets.
Measure more than annotation volume
Counting completed samples doesn't show how much time annotators spend on their work, how often labels change, or how frequently submissions are rejected during review. Voxel51's new metrics page tracks four areas of annotation activity:
- Throughput: See how many samples each person annotates and track sample progress over time.
- Time spent: Examine average time spent on samples and total annotation time, including information useful for billing.
- Label activity: Count labels added, deleted, and modified.
- Review outcomes: Track approval and rejection rates to see how often submitted samples are approved or rejected.
These measurements let teams examine annotation output alongside the effort involved and the results of review.
For example, if rejection rates rise, a team can compare review outcomes with label modifications and average time spent on samples. That comparison can help identify patterns worth investigating. The metrics show what happened across a dataset and now tell you drill down on why a particular label was rejected.
Examine annotation metrics across workflows and datasets
Annotation performance can look different at the individual, dataset, and sample levels. Voxel51 lets teams examine metrics for an individual workflow, all workflows within a dataset, or datasets across a deployment. They can also break down activity by individual annotator and workflow stage.
The metrics distinguish between the Annotate and Review stages, with measures tailored to each stage's activities. This allows teams to examine annotation work and review outcomes in the context of the workflow where they occur.
Metrics for Annotate and Review stages
| Metric | Annotate | Review |
|---|---|---|
| Samples | ✅ | ✅ |
| Labels touched | ✅ | ➖ |
| Labels reviewed | ➖ | ✅ |
| Added | ✅ | ➖ |
| Deleted | ✅ | ➖ |
| Modified | ✅ | ➖ |
| First-pass approval | ✅ | ➖ |
| Rejection rate | ✅ | ✅ |
| Total time | ✅ | ✅ |
| Avg time / sample | ✅ | ✅ |
Voxel51 records actions performed in the application through an event system, then aggregates those events into the metrics displayed on the page. This provides a consistent way to summarize activity across the supported workflow levels.
Pinpoint issues with spatiotemporal comments
Metrics show patterns across annotation work, but reviewers still need a precise way to communicate problems with individual samples.
Voxel51 adds spatiotemporal comments that reviewers can attach to specific locations in images and specific frames in video and multimodal datasets.
For example, instead of telling an annotator that a video annotation is incorrect without identifying where, a reviewer can now attach feedback to the relevant frame. For an image, the reviewer can tie the comment to the location that needs attention. The annotator gets a specific reference for addressing the feedback.
Get started
Explore annotation metrics and spatiotemporal comments in Voxel51 to inspect annotation activity, review outcomes, and feedback in context. Try Voxel51 for free, with just an email signup.
FAQ
Voxel51 annotation metrics cover four areas: throughput, time spent, label activity, and review outcomes. Together they show how much work each annotator completes, how long it takes, how labels change, and how often submissions pass review.
Yes. Voxel51 breaks down annotation activity by individual annotator and by workflow stage, with separate measures for the Annotate and Review stages.
Voxel51 records actions in the app through an event system, then aggregates those events into the metrics shown on the metrics page.
Spatiotemporal comments are reviewer feedback tied to a specific location in an image or a specific frame in a video or multimodal dataset. They give annotators a precise reference for fixing a label.
Founding Machine Learning Engineer
Degrees in Physics, Computer Science, and Machine Learning, but I primarily spend my time as an ML engineer focused on Computer Vision. Recent interests have ranged from object detection, to video understanding, to data visualization.
See all articles by Eric Hofesmann
Try Voxel51 for yourself
Loading related posts...
© 2026 Voxel51 All Rights Reserved
