Model evaluation measures how well a trained model performs on data it hasn't seen, and, more usefully, where and why it fails. This guide covers model evaluation for visual and physical AI: which metrics fit classification, detection, segmentation, and robot policies, why aggregate scores hide the failures that matter, how leaky test sets and benchmarks inflate results, and how teams turn evaluation failures into the next round of data fixes.
What is model evaluation for visual and physical AI?
Machine learning model evaluation is the process of measuring a trained model's predictions against ground truth on held-out data, to decide whether the model is good enough to ship and what to fix if it isn't. For visual AI, that means comparing predicted labels, boxes, and masks against annotated images and video. For physical AI, it also means measuring whether a robot policy completes its task when it runs, in simulation or on hardware.
Evaluation has two jobs, and many teams stop at the first. The first is scoring: a number such as accuracy, mean average precision (mAP), or success rate that summarizes performance. The second is diagnosis: finding the specific samples, scenarios, and episodes where the model fails, and tracing each failure to its cause. A single score tells a team that something is wrong. It rarely tells them what.
Evaluation is also only as trustworthy as the data it runs on. Test sets with label errors, near-duplicates of training samples, or too few examples of rare conditions produce confident numbers that don't hold up in production. Evaluating the evaluation data is part of the job.
Model evaluation works best as a loop rather than a final exam. Teams pick metrics that fit the task, score the model, slice the results by scenario, inspect the failing samples, fix the data behind them through curation and annotation, retrain, and evaluate again. The diagram below summarizes the loop, and the rest of this guide follows it.
Model evaluation as a loop: score, slice, inspect the failing samples, fix the data, and evaluate again.
Model evaluation measures a model's predictions against ground truth on held-out data, and the useful output is a list of failures, not only a score.
The right model evaluation metrics depend on the task: precision, recall, and F1 score for classification, intersection over union (IoU) and mean average precision (mAP) for detection, mean IoU (mIoU) and the Dice coefficient for segmentation, and success rate for robot policies.
In model evaluation, aggregate metrics hide failures. Data slices and sample-level review show which scenarios, classes, and conditions a model gets wrong.
Robot policy evaluation rests on success rates from small numbers of rollouts, so confidence intervals, perturbation tests, and rollout review matter more than a headline number.
Model evaluation tutorials and guides from the Voxel51 blog
These posts cover model evaluation from per-class metrics to robot policy failures. Each section below links to more.
Model evaluation draws its vocabulary from statistics, benchmarking, and robot learning. Metrics are defined where they come up below. These six terms anchor the rest of this guide.
Ground truth: the labels accepted as correct, the reference a model is trained and measured against.
Data slice: a subset of evaluation data defined by a condition, such as night scenes or small objects, scored separately from the whole.
Error analysis: the systematic review of a model's mistakes to find patterns and causes.
Failure mining: searching a dataset for every sample or episode similar to a known failure.
Model calibration: how closely a model's confidence scores match how often it is actually right.
Policy evaluation: measuring how often a robot policy completes its task, usually as a success rate over rollouts.
Which model evaluation metrics fit which task?
Every metric answers a narrower question than "is the model good?" The task decides which questions matter. The table below maps the common visual and physical AI tasks to their standard metrics.
Detection metrics, with IoU computed on masks instead of boxes
Robot policies
Success rate, with confidence intervals
How often the policy completes the task, and how certain that estimate is
Model evaluation metrics by task, with what each set of metrics answers.
Two choices trip up teams more than any formula. The first is accuracy on imbalanced data. When 99% of samples are negative, a model that predicts "negative" every time scores 99% accuracy and finds nothing, which is why precision, recall, and F1 are the default for rare classes. The second is ROC versus precision-recall curves. On imbalanced data, the ROC curve can be misleading because its baseline stays at 0.5 no matter how rare the positives are, while the precision-recall baseline moves with prevalence. Precision-recall curves are usually the better view when positives are rare, and in detection, where true negatives (background) aren't even defined.
Object detection evaluation metrics
Object detection is scored in two steps. First, each predicted box is matched to a ground truth box if their IoU, the overlap divided by the union, clears a threshold. Then precision and recall are computed from the matches across confidence scores, and the area under the resulting precision-recall curve is the average precision for that class.
The COCO benchmark made one variant the standard. Its primary metric averages AP over 10 IoU thresholds from 0.50 to 0.95 in steps of 0.05, which rewards boxes that fit tightly, and then averages across all 80 categories. COCO makes no distinction between AP and mAP. AP50, the older PASCAL VOC metric, uses a single 0.50 threshold and is the more lenient of the two. COCO also reports AP separately for small, medium, and large objects, which is often the first place a detector's weaknesses show up.
Even a strong mAP hides which kind of error dominates. COCO's own error analysis of the 2015 challenge-winning ResNet detector found that AP at an IoU of 0.75 was 0.399, and that perfect localization alone would raise it to 0.682. Its errors were dominated by imperfect localization and background confusions, and each of those calls for a different fix than missed objects do.
IoU is the overlap divided by the union. A prediction counts as a match only when its IoU clears the threshold, and COCO's primary metric averages AP over thresholds from 0.50 to 0.95.
Calibration and confidence thresholds
Most classifiers and detectors output a confidence score, and production systems act on a threshold. That only works if the scores mean what they say. Calibration measures the match between confidence and correctness, usually as expected calibration error (ECE), the weighted average gap between confidence and accuracy across confidence bins. Guo et al. (2017) found that modern classification networks are poorly calibrated, usually overconfident, and that temperature scaling, a single-parameter fix, is surprisingly effective. Detector scores are often not probabilities at all, which makes calibration harder to check. Either way, a threshold chosen on an uncalibrated model can move a long way when the model is retrained.
Why aggregate metrics hide failures: failure analysis with data slices
A single score averages over everything, so it's dominated by the common cases. The rare conditions that cause production incidents, such as night, glare, occlusion, unusual objects, and edge-of-frame targets, make up a small share of the test set and barely move the average. A model can improve overall while getting worse on exactly the cases a team cares about. In a COCO example from Voxel51's post on the long tail of autonomous driving, the detector with the higher overall mAP (0.396 versus 0.381) trailed on the rare classes.
Data slices fix this by scoring subsets separately. A slice can come from metadata (weather, camera, location, time of day), from labels (object size, class, occlusion), or from the data itself (clusters in embedding space). Slicing turns one number into a table of numbers, and the outliers in that table are the failure modes. Small slices carry wide error bars, the same problem as small rollout counts in robot evaluation below, so a slice with 30 samples needs a bigger gap before it counts as a failure. When the slices are scenarios that matter to the product, such as "pedestrian at dusk" or "reflective packaging," the practice is often called scenario-based evaluation, and searching data for those scenarios is scenario mining.
A higher overall score can hide a worse model on the slices that matter. Scores are illustrative.
Slices find where a model fails. Sample-level review finds why. Looking at the actual false positives and false negatives in a failing slice usually sorts them into a few causes: a real model weakness, a gap in the training data, or a wrong label in the test set. Each has a different fix, and only sample-level review tells them apart. The worked scenario in Voxel51's post on sample-level debugging shows the pattern: a model with 94.2% overall accuracy that fails badly on rainy images, a slice that makes up 15% of production traffic.
In Voxel51, the Model Evaluation panel connects these levels. Teams see classification reports, confusion matrices, and precision-recall curves, click a cell or a bar to filter to the samples behind it, and review predictions against ground truth on the images themselves. Scenario analysis scores models across slices the team defines, such as scene type, weather, or object size.
Scenario analysis scores two models category by category, so a model that wins overall can be checked slice by slice.
Comparing models beyond a single score
Comparing two models on one number invites the same mistake. A new model that wins on mAP can lose on a slice that matters, and a small overall gain can be noise. Comparing models per slice, and looking at the samples where they disagree, shows whether an upgrade is safe to ship. In Voxel51, teams compare two or more evaluation runs side by side, and the Voxel51 Agent answers questions such as "show me samples where model A performed worse than model B" in plain language.
Two evaluation runs compared side by side. On the same 2,162 images, PatchCore produces 280 false positives to PaDiM's 430, and each count opens the samples behind it.
Is your test set lying? Data leakage and benchmark contamination
An evaluation score is only as honest as the split behind it. Data leakage in machine learning is any path by which information that won't be available at inference reaches training, from features that encode the label to preprocessing fit on the whole dataset. In visual and physical AI, the dominant form is train-test contamination, where the model is graded partly on examples it has already seen, and that's this section's focus. The result is a score that looks good and doesn't transfer.
Leakage is common and well documented. Barz and Denzler found that 3.3% of CIFAR-10 and 10% of CIFAR-100 test images have near-duplicates in the training set, and that on duplicate-free test sets, error rates rose by 9% to 14% relative to the original ones. Kapoor and Narayanan compiled evidence of leakage across 17 scientific fields, affecting 329 papers. Test labels have their own problems. Northcutt et al. estimated an average of at least 3.3% label errors across 10 widely used test sets, enough, in their analysis, to flip model rankings on noisier test sets.
Visual and physical AI data leaks in a specific way. Video frames, driving logs, and robot episodes are sequences, and adjacent frames are nearly identical. Splitting at the frame level puts near-copies of the same moment on both sides of the split. The fix is to split by the unit that is actually independent: the sequence, the drive, the episode, the scene, or the location. Near-duplicate detection over embeddings catches what a metadata split misses.
Frame-level splits leak near-identical frames into the test set. Split by episode, sequence, or scene instead.
Public benchmarks add a second layer. Even a faithful copy of a test set can move the score. When Recht et al. rebuilt the CIFAR-10 and ImageNet test sets following the original collection process, accuracy dropped by 3 to 15 percentage points, and the authors traced the drop to models failing on slightly harder images rather than to overfitting the benchmark. Memorization does happen when a benchmark's test layouts match its training demonstrations, as the LIBERO results below show. Either way, a benchmark score is evidence about the benchmark first.
How do you evaluate a robot policy? Physical AI model evaluation
Physical AI model evaluation asks a different question from perception metrics: did the robot do the task? A policy is scored by running it, in simulation or on hardware, and counting successes. Each run is a rollout, and the standard metric is the success rate, the share of rollouts that achieve the goal.
Success rate is simple and often misleading. A 2024 best-practices paper on policy evaluation noted that success rates are frequently reported with little or no information about the number of runs, the initial conditions, or the success criteria, and with little or no statistical analysis. Real-world rollouts are slow and expensive, so trial counts stay small, and small counts carry wide error bars. As Voxel51's sim-to-real post puts it, nine successes in ten rollouts is not a 90% policy. The exact 95% confidence interval runs from about 56% to 99.7%. Ninety in a hundred, a realistic lab count, still spans about 82% to 95%. Nine hundred in a thousand narrows it to roughly 88% to 92%.
Rollouts
Successes
Observed success rate
Exact (Clopper-Pearson) 95% confidence interval
10
9
90%
55.5% to 99.7%
100
90
90%
82.4% to 95.1%
1,000
900
90%
88.0% to 91.8%
The same 90% success rate from 10, 100, and 1,000 rollouts, with the exact confidence interval for each.
Teams stack methods to get more signal per real rollout. Offline review of logged runs is cheapest, simulation scales, scenario tests target known weak spots, and real-world trials are the final check. Simulation only helps if it predicts reality. SIMPLER showed that simulated environments built to close the control and visual gaps can correlate strongly with real-world results, which is the property a sim benchmark has to earn. Offline metrics don't substitute for rollouts either. In one 2026 study covered in Voxel51's post on robot policy training data, validation loss was uncorrelated with real-world success (r = −0.04) and even rose while performance improved, while training loss (−0.84) and validation action error (−0.89) did track it. The authors still conclude that offline metrics can't reliably predict success rates or reveal failure modes.
Nine of ten is not a 90% policy. The width of each interval is what a small rollout count can't tell you.
Why LIBERO benchmark scores mislead
LIBERO is a simulation benchmark of 130 language-conditioned manipulation tasks, and four of its 10-task suites (Spatial, Object, Goal, and Long) have become the standard scoreboard for vision-language-action (VLA) models. Headline scores above 90% are now common, and they say less than they appear to. LIBERO-PRO perturbed objects, initial states, instructions, and environments, and found that models scoring over 90% on standard LIBERO collapsed to 0.0%, a sign of memorized action sequences rather than learned skills. LIBERO-Plus found performance dropping from 95% to below 30% under modest changes such as camera viewpoint and robot initial state, and that models often ignored the language instruction entirely.
Scores also don't compare cleanly across papers. As Voxel51's VLA comparison shows, the same model, π0, is reported at 86.0 in one paper and 94.2 in another, because evaluation setups differ. A LIBERO number is a starting point for questions, not a ranking.
Reviewing rollouts: where policy failures come from
Success rates say how often a policy fails. Rollout review says how. Teams record evaluation rollouts with the same sensors as training data, then watch the failures, tag where they start, and group them by pattern, such as grasp slips, wrong-object picks, or timeouts. In Voxel51, teams review recorded rollouts as synchronized multimodal episodes, mark failure moments with temporal tags, and search for similar moments with segment-level embeddings. In Voxel51's failure mining walkthrough on 100 episodes from the DROID robot manipulation dataset, one dropped-cloth failure surfaced 16 similar episodes, and filtering to failed episodes narrowed that to 2. Those episodes become the next round's training and evaluation targets.
Evaluation pays off when its failures change the next training set or the model. Failures found by sample-level review often trace back to one of a few causes, and each has a standard fix.
What review finds
Likely cause
Fix
The model fails across many conditions, including well-covered ones
Model weakness
Change the architecture, capacity, or training recipe
What sample-level review finds, the likely cause, and the fix for each.
Label errors are easy to miss because they corrupt the score and the training signal at the same time. For coverage gaps, the fix is targeted: find every sample like the failure, label those, and retrain, instead of labeling more of everything. Active learning generalizes the idea by letting model uncertainty pick the next samples to label. In Voxel51, teams curate training data from model failures and route the selected samples to annotation, so the next evaluation measures the fix.
Ancera's team, for example, found model issues by reviewing false positives, false negatives, and confusion matrices, relabeled the affected detections, and reported a more than 7% improvement with gains in F1, precision, and recall.
The loop is the same across domains. What changes is the metric that matters, the conditions that break models, and the cost of a missed failure. The physical AI guide covers robotics and autonomy in depth.
Autonomous driving
Driving models are scored on detection and tracking, but the risk sits in the long tail of rare classes and conditions that aggregate mAP averages away.
Robot evaluation combines perception metrics with policy success rates, and investigating a failure means replaying every sensor at the moment it happened, such as a wrong suction cup grasping an item.
Camera-based detection lives or dies on false positives under real conditions such as glare, reflections, and lighting changes. Vivint evaluates its models across lighting, weather, and camera types and traces false positives to the samples behind them. Protex AI compares model versions side by side while staying grounded in the images.
Standard per-image metrics don't always match the job. IBAK computes IoU and mAP per image but also built a report-level metric for pipe inspection: did the model find every defect along the full length of the pipe?
In safety-critical detection, the cost of a miss shapes the metric. SafelyYou uses evaluation to find both model failures and annotation mistakes in fall detection video.
You can run the metric and failure analysis steps in this guide without writing code. In the free tier of Voxel51, you can:
Browse the 20+ demo datasets and add VisA Anomaly Detection, 2,162 industrial inspection images with evaluated predictions from two anomaly detection models, PaDiM and PatchCore. Each dataset, demo or your own, uses one of three free dataset slots.
Open the Model Evaluation panel, read each model's metrics, and click a true positive, false positive, or false negative count to see the samples behind it.
Compare the two models side by side, or ask the Voxel51 Agent to "show me samples where the PaDiM model performed worse than the PatchCore model."
Tag the failing samples and save them as a subset to curate or relabel.
For classification, the Chest X-Ray demo pairs 10,000 NIH chest X-rays with an evaluated classifier's predictions.
Model evaluation for visual and physical AI scores predictions on images, video, point clouds, and robot episodes, so the metrics are spatial (IoU, mAP, mIoU) or behavioral (policy success rate) rather than text-based. Test data is often sequential, which makes leakage between frames a bigger risk, and the failures that matter tend to sit in rare conditions such as night, glare, or unusual objects, so slicing and sample-level review carry more weight than a single score.
Average precision (AP) is the area under the precision-recall curve for one class. Mean average precision (mAP) averages AP across classes. On the COCO benchmark, AP is also averaged over IoU thresholds from 0.50 to 0.95, and COCO uses the two terms interchangeably.
A precision-recall curve is usually the better choice when positives are rare, which covers most visual AI detection and defect tasks. The ROC curve's baseline stays fixed regardless of class balance, so it can look strong on imbalanced data even when the model finds few positives. ROC-AUC is reasonable when classes are roughly balanced or true negatives matter.
Data slices expose model failures by scoring subsets of the evaluation data separately, such as night scenes, one camera type, or small objects. A failure that an aggregate metric averages away shows up as an outlier slice, which tells the team where to look at individual samples.
Data leakage in machine learning is any path by which information that won't be available at inference reaches training, such as features that encode the label, preprocessing fit on the whole dataset, or test samples that also appear in training. It inflates evaluation scores. In visual and physical AI, a common cause is splitting video frames or robot episodes at the frame level, so near-identical frames land in both training and test sets.
Robot policy evaluation runs the policy in simulation or on hardware and reports the success rate across rollouts, ideally with the number of rollouts, the success criteria, and a confidence interval. Teams then review the failed rollouts to find patterns, and use perturbed or held-out conditions to check that the policy generalizes.
LIBERO scores mislead because models can memorize the benchmark's fixed layouts and action sequences. Under perturbations to objects, layouts, camera views, or instructions, models scoring over 90% on standard LIBERO have dropped to near zero, and the same model is reported with different scores in different papers.