Aggregate metrics lie by omission. "Mean Dice 0.82, ship it" hides the truth: your model nails big, high-contrast organs and quietly falls apart on the small, variable ones. Pair TotalSegmentator with FiftyOne and that blind spot becomes something you can see — the exact patient, the exact slice, the exact organ where it breaks.
FiftyOne App showing a CT scan with TotalSegmentator organ masks overlaid in axial, coronal, and sagittal views.
What is TotalSegmentator?
TotalSegmentator is the open-source workhorse of medical image segmentation. From a raw CT or MR volume it labels over 100 anatomical structures — organs, bones, vessels, muscles — with no per-organ training on your end. It's built on nnU-Net, runs on Ubuntu, macOS, and Windows (CPU or GPU), and is robust across scanners, institutions, and protocols because it was trained on exactly that kind of variety.
As described in a Radiology: Artificial Intelligence paper, it ships subtasks for everything from body composition to vertebrae, and is genuinely good enough to sit underneath real clinical research pipelines. One pip install, one command, and you have segmentations.
Key takeaways
Aggregate metrics like mean Dice hide failure. A 0.82 average can mean strong performance on large organs and poor performance on small ones.
TotalSegmentator labels over 100 anatomical structures from a raw CT or MR volume, with no per-organ training required.
FiftyOne pairs predictions with ground truth so you can sort by accuracy, inspect the worst cases, and see exactly where a model breaks.
Some "worst predictions" are bad ground truth, not model errors, so the same workflow doubles as a data audit.
Both tools are open source and install in minutes.
What is FiftyOne?
FiftyOne is an open-source tool from Voxel51 for actually looking at your data and model predictions — instead of trusting a summary row in a table. Load images, videos, or volumes; overlay labels and predictions; then slice, filter, sort, and evaluate your dataset from Python or an interactive App.
Its whole reason for existing is the gap between "the metric says the model is fine" and "the model is not fine." FiftyOne traces poor metrics down to the individual sample, surfaces edge cases, and helps you build better datasets — which, more often than better architectures, is what actually moves model quality.
Go from “looks fine” to “there's the bug” in ten seconds"
Run TotalSegmentator over a stack of CT scans. Load the predictions and the ground truth into FiftyOne side by side. Then:
Sort by accuracy, worst first
The model's failures float to the top of the grid. Open one. Toggle prediction against ground truth. Watch it erode a kidney boundary or drop the gallbladder entirely — in full anatomical context.
No spreadsheet. No guessing. Just the failure, right there.
The things you only catch by looking
Table ranking TotalSegmentator structures by F1, liver and aorta highest, pancreas and adrenal glands lowest.
Per-structure truth
Rank every organ by IoU and the real story appears: liver and aorta up top, pancreas and adrenal glands at the bottom. That gradient is the finding.
Metric-proof bugs
A vertebra labeled L2 instead of L3 barely moves Dice — the pixels are in the right place, just wrongly named. Color each vertebra by an ordered label and the mix-up jumps out as a broken rainbow. FiftyOne catches identity errors that numbers hide.
Label noise, exposed
Some of your "worst predictions" aren't the model's fault — they're bad ground truth. Finding those flips the whole exercise: you're not just grading the model, you're auditing the data.
Smarter curation
Diversity-aware sorting surfaces a spread of failures across anatomy and patients, not 30 near-identical bad slices — so your re-annotation budget goes where it counts.
Metrics tell you that a model is wrong. TotalSegmentator and FiftyOne together tell you where, why, and what to fix — in the time it takes to sort a grid. Both are open source. Both install in minutes. Go break your model on purpose.
An open-source medical image segmentation tool that labels over 100 anatomical structures (organs, bones, vessels, muscles) from a raw CT or MR volume. It is built on nnU-Net, runs on Ubuntu, macOS, and Windows on CPU or GPU, and needs no per-organ training.
You load predictions and ground truth side by side, then sort, filter, and toggle overlays to find the exact patient, slice, and organ where segmentation breaks, instead of trusting an aggregate score.
A high average is dominated by large, high-contrast organs. Small, variable structures can fail badly while barely moving the mean, so the headline number looks fine even when the model is not.
Small, low-contrast structures such as the pancreas, adrenal glands, and gallbladder rank lowest, while the liver and aorta rank highest.
Yes. Some of the worst-scoring predictions are actually bad ground truth. Surfacing them turns the exercise into a data audit, and identity errors (for example a vertebra labeled L2 instead of L3) show up visually even when they barely move Dice.
