Imitation learning is only as good as the demonstrations you feed it. Train a policy on 6,500 robot episodes and a handful of botched grasps, dropped objects, or glitchy renders will quietly poison the result — and you'll never see them, because nobody scrubs through thousands of clips by hand. This post walks through a notebook that turns that needle-in-a-haystack problem into a few minutes of visual, searchable curation, by pairing the LIBERO robotics benchmark with FiftyOne.
Grid of sampled frames from LIBERO robot manipulation episodes in the FiftyOne App
Key Takeaways
LIBERO is a simulation benchmark of 130 language-conditioned robot manipulation tasks, teleoperated on a Franka Emika Panda arm and published in LeRobot format on the Hugging Face Hub.
FiftyOne loads LIBERO episodes as grouped multi-camera samples, so agent and wrist camera views play back in sync.
CLIP embeddings of LIBERO frames power both UMAP cluster visualization and natural-language search over demonstrations, no labels required.
FiftyOne's uniqueness scoring surfaces bad LIBERO demonstrations, like dropped objects and rendering glitches, before they poison policy training.
What Is LIBERO? A Robotics Benchmark Built for Exploration
LIBERO — LIfelong learning BEnchmark on RObot manipulation — is a simulation benchmark for studying how robots transfer knowledge across tasks. A few things make it a great dataset to explore rather than just train on:
It's language-conditioned and structured.
LIBERO ships 130 manipulation tasks grouped into four suites, each isolating a different kind of generalization: Spatial (same objects, different arrangements), Object (same layout, different objects), Goal (fixed objects and layout, different target behaviors), and LIBERO-100/Long (long-horizon tasks that entangle all three). Every episode carries a natural-language instruction like "put the bowl on the stove" — which, as you'll see, becomes a powerful search key.
Every demonstration is multi-view.
Each episode is recorded from a third-person agent camera and a wrist-mounted camera, plus the robot's proprioceptive state and delta actions at every frame. That's rich, aligned, multimodal data.
Real teleoperated data at scale.
The demonstrations are human-teleoperated on a Franka Emika Panda arm in MuJoCo — high-quality, but human, which means variation and the occasional bad take. Exactly the kind of thing worth inspecting before training.
Because LIBERO is published in the LeRobot format on the Hugging Face Hub, it drops straight into modern robotics tooling — including FiftyOne.
LIBERO's four task suites, each isolating a different kind of generalization.
LIBERO's four task suites, each isolating a different kind of generalization.
Suite
What varies
Generalization tested
Spatial
Same objects, different arrangements
Spatial reasoning
Object
Same layout, different objects
Object-level transfer
Goal
Fixed objects and layout, different target behaviors
Goal/behavior transfer
LIBERO-100/Long
Long-horizon tasks entangling all three
Long-horizon transfer
How popular is LIBERO?
LIBERO has become one of the de facto standard benchmarks for vision-language-action (VLA) research. On the Hugging Face Hub it consistently shows up among the trending robotics datasets, and the LeRobot re-release has hundreds of models trained on it — a stronger signal than raw download counts, since it means the dataset is actually being built on, not just grabbed once. That puts it in rare company: the vast majority of the Hub's 90,000+ image datasets are used once and forgotten, while LIBERO appears again and again in papers reproducing and comparing policy-learning results (Physical Intelligence's π0.5, for instance, reports LIBERO benchmark numbers). For robotics specifically it sits alongside datasets like DROID and BridgeData as a go-to reference — and unlike large real-world corpora, its clean simulated structure makes it especially pleasant to explore.
FiftyOne: A Microscope for Visual Datasets
FiftyOne is an open-source toolkit for annotating, visualizing, exploring, and curating visual datasets. Instead of writing one-off scripts to eyeball your data, you load it into a FiftyOne Dataset, launch an interactive App in your browser, and slice, filter, search, and annotate from there. Its "Brain" methods add the parts that matter for large datasets: computing embeddings, projecting them into interactive 2D plots, powering similarity search, and scoring samples for uniqueness and likely mistakes. It handles images, videos, 3D, and — via grouped datasets — multi-camera scenes where several views belong to one logical sample.
That last capability is what makes it a natural fit for robotics data, where a single episode is several synchronized video streams.
Turning 6,500 Demos Into Something You Can Reason About
The notebook loads a slice of LIBERO into FiftyOne and turns a pile of clips into something you can actually reason about. The combination shines in a few specific ways:
Episodes become grouped, multi-camera samples.
Using a community LeRobot importer, each LIBERO episode lands as a group with the agent and wrist cameras as slices. In the App you flip between camera views with a dropdown, and playback stays synchronized — a folder of MP4s can't do that.
CLIP embeddings reveal the structure of the benchmark.
The notebook computes CLIP embeddings over sampled frames and projects them with UMAP. Color the scatterplot by task and the suites cluster visually — demonstrations of the same instruction land together. Lasso a cluster and the sample grid filters to match, live.
Natural language becomes a query
Because CLIP is multimodal, the same embeddings power free-text search. Type "a robot arm reaching toward a microwave" and the most relevant demonstrations sort to the top — no labels required.
Bad demos surface themselves with uniqueness scoring
A uniqueness score ranks every frame by how visually distinct it is. High-uniqueness frames are the outliers — odd arm poses, dropped objects, rendering glitches — the candidate bad demonstrations. The notebook rolls those frame-level findings up to the episode level and tags the offending episodes for review, so "find the weird demos before you train on them" becomes a two-minute pass instead of an afternoon.
LIBERO gives you richly structured, multi-view, language-labeled robot data, and FiftyOne gives you the lens to curate it — visually, semantically, and at scale.
Next steps
Run the notebook.Download it from GitHub. It's self-contained, targets macOS with the latest FiftyOne, and includes setup for an isolated virtual environment. Bump MAX_EPISODES and the workflow is identical at full scale.
Go deeper: repeat the embedding analysis on the wrist camera and diff the clusters, or put a vision-language model in the loop to flag episodes where the caption disagrees with the task instruction.
Robotics data is exploding, and most of it never gets looked at. Tools like FiftyOne make "actually look at your data" cheap enough to do routinely.
FAQ
LIBERO (LIfelong learning BEnchmark on RObot manipulation) is a simulation benchmark of 130 language-conditioned manipulation tasks grouped into four suites, built to study how robots transfer knowledge across tasks.
Use the community LeRobot v3.0 importer, which lands each episode as a group with the agent and wrist cameras as slices, keeping playback synchronized.
Compute a uniqueness score for every frame, then review the high-uniqueness outliers, which flag odd arm poses, dropped objects, and rendering glitches, rolled up and tagged at the episode level.
Yes. Because CLIP embeddings are multimodal, a free-text query like "a robot arm reaching toward a microwave" sorts the most relevant LIBERO demonstrations to the top.
