In Part 1 we put the Community Fish Detector under the FiftyOne microscope — one model, 17 underwater environments, and a per-source mAP table that showed exactly where "find any fish, anywhere" holds up. Now we point the same notebook at a completely different kind of fish data: not blurry footage from the wild, but pristine museum specimens in a drawer. Same tool, new modality — and one genuinely surprising result at the end.
FiftyOne grid of Fish-Vista museum fish specimens in natural color, each tagged with species, family, and source
Key Takeaways
Fish-Vista is ~60,000 museum specimen images across ~1,900 species, with pixel-level trait segmentation of 9 anatomical structures (eye, head, barbel, and the dorsal, adipose, pectoral, pelvic, anal, and caudal fins).
Where Part 1 stressed detection and domain gaps, Part 2 exercises segmentation, long-tail taxonomy, and multi-label traits, a different set of FiftyOne muscles on the same fish.
Trait presence becomes multi-label tags for free, read straight from the masks, so "show me every fish with a barbel" needs zero extra annotation.
FiftyOne turns ~1,900 imbalanced species into a grid you can click into, singletons and all, so the long tail stops being a footnote.
Run cold on the museum specimens with no fine-tuning, Part 1's wild-trained detector found the fish in 100% of images at ~0.97 mAP, least confident on oddballs like scorpionfish but never actually missing one.
The whole demo streams a curated subset from Hugging Face, no 11.9 GB clone, laptop-friendly.
Meet Fish-Vista: a 60,000-image museum specimen dataset
Fish-Vista is the opposite of CFD's footage. These are natural-history specimen photos — one fish, centered, lateral view, on a clean white background — curated from museum collections (GLIN, iDigBio, Morphbank) via Fish-AIR. It supports species classification, trait identification, and pixel-level trait segmentation of nine anatomical parts.
CFD asked a detection question: can one model find fish across murky, wildly different environments? Fish-Vista asks an understanding question: can we see and quantify fish anatomy and taxonomy at scale? Different question, different FiftyOne toolkit.
What FiftyOne reveals in a drawer full of fish
Anatomy you can actually see.
Load the segmentation masks and every specimen wears its parts in color — eye, head, barbel, and each fin — with trait names on hover. Filter the legend to one trait and scan it across species: comparative anatomy as a scrollable grid.
Traits from masks, for free.
Because "which parts are present" is read directly off the pixels, you get ground-truth multi-label attributes with no extra labeling. That powers compound queries — barbel AND adipose fin, missing pelvic fin, most anatomically complete — all consistent with the mask by construction.
A long tail you can click.
Fish-Vista is deliberately imbalanced across ~1,900 species. FiftyOne's distributions plus a "rarest species" saved view drop you straight into the one-off classes that quietly wreck classifier accuracy.
Embeddings that map morphology.
On clean, uniform specimen photos, embeddings are organized by body plan. Color the plot by family and taxonomy emerges as clusters; lasso one to pull up lookalikes, or run similarity search to find "more fish shaped like this."
A built-in data-quality audit.
Every specimen carries its source museum, owning institution, and license. Slice by source to compare imaging conventions, flag sparse or broken masks by coverage, and surface near-duplicate specimens with uniqueness — a redistribution-aware "clean before you train" pass.
The twist: a wild-fish detector, loose in the museum
Here's the part we didn't expect. We took Part 1's CFD detector — trained only on murky in-situ footage — derived a whole-fish ground-truth box from each specimen's trait mask, and ran the detector cold on the museum images.
It didn't break. In our run it found the fish in 100% of specimens at ~0.97 mAP, with mean top confidence around 0.96. The domain shift ran in the favorable direction: a centered, high-contrast fish on white is easier than a half-occluded one in turbid water.
Dot plot of the wild-trained CFD detector's mean top confidence per fish family on Fish-Vista, lowest for scorpionfish (0.885)
The residual is where it gets fun. Break confidence down by family and a sensible morphology ranking falls out: the detector is least sure about scorpionfish (Scorpaenidae) — the spiky, camouflaged, least-fusiform fish in the drawer — followed by elephantfish and gobies. But here's the kicker: its miss rate across every one of those families is zero. Least confident ≠ wrong. A handful of specimens also drew a second box, almost always a scale bar or specimen tag the model read as a bonus fish.
Mean top confidence of the wild-trained CFD detector run on Fish-Vista museum specimens, by fish family. Miss rate was 0% across every family; overall mean confidence was 0.957. Scorpionfish (Scorpaenidae), the spiky, camouflaged, least-fusiform family, scored lowest yet were still detected.
Mean top confidence of the wild-trained CFD detector run on Fish-Vista museum specimens, by fish family. Miss rate was 0% across every family; overall mean confidence was 0.957. Scorpionfish (Scorpaenidae), the spiky, camouflaged, least-fusiform family, scored lowest yet were still detected.
Family
Specimens (n)
Mean top confidence
Blenniidae
8
0.963
Cyprinidae
11
0.962
Acanthuridae
9
0.962
Carangidae
3
0.962
Bythitidae
3
0.962
Serranidae
13
0.961
Kyphosidae
4
0.960
Mormyridae
4
0.957
Holocentridae
9
0.956
Chaetodontidae
4
0.955
Gobiidae
12
0.954
Pinguipedidae
4
0.952
Pimelodidae
3
0.951
Pseudochromidae
3
0.931
Scorpaenidae
8
0.885
Read that as a robustness result: a field-trained detector can pre-annotate an entirely different imaging modality with no fine-tuning. FiftyOne is where you prove that — evaluate the cross-domain run, break confidence down by family, and click straight to the one specimen worth a second look.
Fish-Vista is a dataset of ~60,000 museum specimen images spanning ~1,900 fish species, curated from museum collections (GLIN, iDigBio, Morphbank) via Fish-AIR. It supports species classification, trait identification, and pixel-level trait segmentation of nine anatomical parts.
CFD is in-the-wild footage for a single-class detection task across 17 capture environments. Fish-Vista is ~60,000 clean museum specimen images across ~1,900 species, built for classification, trait identification, and pixel-level trait segmentation. Different imagery, different tasks — which is exactly why it exercises different parts of FiftyOne.
Segmentation labels each anatomical structure at the pixel level — eye, head, barbel, and six fins. Beyond the striking overlay, it lets FiftyOne derive multi-label trait tags directly from the masks, so you can filter by anatomy ("has a barbel," "missing a pelvic fin") without any extra annotation.
In our run, yes — remarkably well. Running the CFD detector on Fish-Vista with no fine-tuning found the fish in 100% of specimens at roughly 0.97 mAP. The shift is favorable (specimens are easier to detect than murky footage), so the result reads as cross-domain robustness and a viable auto-annotation shortcut, not a failure case.
No. The notebook streams a curated subset per-file from Hugging Face — no 11.9 GB clone and no Git LFS — so you can explore masks, embeddings, and the cross-domain probe on a laptop.
Fish-Vista is released under CC-BY-NC (individual images vary), so treat it as research / non-commercial. The demo deliberately skips the non-redistributable CC-BY-ND subset when curating its sample.
