DROID Dataset in Voxel51: Explore 1,000 Episodes, No Code

Free with a work or .edu email address. Nothing to install.

Key takeaways

The DROID dataset is a 76k-episode robot manipulation dataset collected across 13 institutions, and a 1,000-episode slice of it is open to explore for free in Voxel51 with a work or .edu email address. Each episode plays back as three synchronized cameras plus robot state on one timeline.
In about twenty minutes and with no code, you can learn four things about this DROID dataset slice that you'd otherwise write code to find, and each one changes a training decision. Two labs make up 37% of the slice, which affects how you split it. The task field is free text, which affects how you stratify. Every episode succeeded, which affects what counts as a hard example. And in the first episode you'll open, the grasp lasts about one second of an eleven-second recording, which affects what unit you search and embed.
In Voxel51, a text search of the DROID dataset slice for a gripper closing on a blue block returned 21 episodes, and 14 came from AUTOLab, a lab that contributes 16% of the slice. Part of that skew is real, because AUTOLab recorded 15 of the 34 episodes whose instruction mentions a block. Part of it is the search, because only one of the 21 instructions asks for a blue block. Objects and scenes can cluster by lab, which is one reason to evaluate on a lab the policy never saw in training.
Voxel51's segment-level embeddings split each DROID episode into 5-second windows so a text search returns the moment rather than the whole episode, and temporal tags, subsets, and Export turn what you find into an episode list you can hand to a training script. You can bring your own MCAP recordings through the Import button, too.

Who this post is for

This post is for anyone who has downloaded a robot learning dataset, watched a handful of episodes, and wondered what was in the other several thousand episodes. If you've worked with LeRobot datasets, Open X-Embodiment, or your own teleoperation logs, you'll recognize the questions. If you're newer to robot learning and want to see what a demonstration dataset contains before you train on one, this is a good place to start. By the end you'll know four things about this slice that you'd otherwise write code to find, and each one changes how you'd set up a training run.

You don't need to install anything. Everything in this tutorial happens in the browser, and every section has a Where to click list that names the exact buttons, in the order you need them. Where a step is easy to miss, there's a note that says so.

If you only have five minutes, do step 1 of Question 1 to add the dataset, then skip to Question 3. It's the one that shows what segment-level search does.

Terms you'll see in this post

Episode: one recorded demonstration, from the moment recording starts to the moment it stops. DROID's paper calls these trajectories. Each one runs from a few seconds to about a minute.

Policy: the model that takes in what the robot sees and outputs what the robot should do next. When this post says "the policy," it means the model you'd train on this data.

Teleoperation: a person steering the robot by hand, here with Meta Quest 2 controllers, while the cameras and sensors record. Every DROID episode was collected this way.

End-effector: the working end of the arm, which on DROID is the gripper. End-effector speed is how fast the hand is moving.

Gripper state: whether the gripper is open or closed, derived from the gripper's position as 0 or 1. That's why it draws as a square wave in Question 1.

MCAP and embeddings are explained where they first come up, in About the DROID dataset and Question 2.

About the DROID dataset

DROID (Distributed Robot Interaction Dataset) was released in 2024 by researchers across 13 institutions, co-led by Alexander Khazatsky and Karl Pertsch. It contains 76,000 successful demonstration trajectories, about 350 hours of interaction, collected over 12 months by 50 data collectors in 564 scenes across 52 buildings. The paper counts 86 tasks, and it defines a task as a distinct verb pulled from the instructions, which will matter later in this post. Every episode uses the same hardware: a Franka Panda arm with a Robotiq 2F-85 gripper, two exterior Zed 2 stereo cameras, and a wrist-mounted Zed Mini, teleoperated with Meta Quest 2 controllers and recorded at 15 Hz. After collection, crowd workers wrote up to three natural-language instructions for each episode. The full release also includes roughly 16,000 episodes the collectors marked as not successful, which don't count toward the 76,000. The DROID authors release it under CC BY 4.0 in RLDS (Reinforcement Learning Datasets), a TensorFlow-based format, and a LeRobot-format conversion is on the Hugging Face Hub as lerobot/droid_1.0.1, about 95,700 episodes and over 400 GB including the unsuccessful ones, with lerobot/droid_100 as a 100-episode sample.

The dataset was built on a specific bet: that scene and task diversity, not just volume, is what makes a manipulation policy generalize. The paper tests that bet by comparing a 7,362-episode subset drawn from only 20 scenes against a same-size random subset spanning many scenes, and the diverse subset produced more robust policies. DROID has since become a standard dataset for vision-language-action (VLA) models. Physical Intelligence, for example, publishes π0-FAST-DROID and π0.5-DROID checkpoints fine-tuned on it, which is the practical reason to know it well before you train on it. If you're fine-tuning a VLA model on DROID, or on anything shaped like it, the diversity claim is worth checking against the slice you're actually using.

The copy in Voxel51 is a 1,000-episode slice of DROID's raw 1.0.1 release, converted to MCAP, the default format for Robot Operating System 2 (ROS 2) recordings since the Iron release. Each episode is one .mcap file carrying the three camera streams, joint state, end-effector pose, actions, a transform tree, and a one-message episode summary. Embeddings were pre-computed on 5-second windows using Qwen3-VL-Embedding-2B, and Questions 2 and 3 use them.

One more thing about the slice: every one of the 1,000 episodes is marked success = true, so this is a dataset of demonstrations that worked, and none of the full release's unsuccessful episodes made it in. With no failures to look for, the interesting episodes in Questions 2 and 3 will be the ones where a grasp was hard, not the ones where it failed.

And what this copy is not. It's 1,000 of 76,000 episodes from 12 of the 13 lab folders in DROID's raw release (the paper counts 18 research labs across 13 institutions), so any distribution you see here belongs to the slice, not to DROID.

The embeddings are computed on camera frames only, and joint state and gripper signals aren't in the vector. When you search for a behavior in Question 3, you're searching for what it looked like on camera, not for a pattern in the joint or gripper signals.

The Python SDK and API keys are part of paid plans, so everything in this post happens in the browser. If you push past the self-serve dataset and usage limits you'll be routed to the sales team, which is also how you'd get the full 76,000 loaded.

Question 1: What does one demonstration in DROID look like?

What we're doing. Before we look at 1,000 episodes we're going to look at one, with every sensor playing at once. A robot demonstration is several streams recorded on one clock: three cameras, joint positions, gripper state, and the commands the operator sent. Everything later in this post, from embedding windows of time to searching for a moment to tagging an interval, works on that shared timeline, so it helps to see the timeline first.

Where to click.

Open the DROID link at the top of this post and sign up, or sign in, with a work or .edu email address. After the usual sign-up steps, you land on a page titled Add dataset to Voxel51 to continue exploring, with a Robotics Droid card. Click Add on the card. A dialog asks you to name and tag the dataset, and the defaults are fine, so click Add dataset. When it finishes loading, which takes about 20 seconds, you're on the Samples tab, the grid of thumbnails we'll work in for the rest of the post. If the Voxel51 Agent panel opened on the right, close it with the X so you have the room. You can also find DROID on your own: click Explore in the top navigation of the Datasets home page, type droid in the search box, and click Add on the card. Explore is a library of openly available datasets you can add to your workspace in one click. The free plan holds three datasets, so if the Add button is locked, remove one from the Datasets page first. A brand-new free workspace starts with one sample dataset, Animal Pose Benchmark, so DROID is your second of three.
Click the first thumbnail in the grid. The episode opens full screen with camera/ext1 and camera/ext2 side by side. On the left is a pane with tabs for Scene and Topics, plus a third tab for whichever tile is selected, camera/ext1 to start. The arrow at the top left takes you back to the grid.
Click Topics. You'll see every stream in the recording with its message rate. Under Sensors, click the Image link next to /camera/wrist/image. A third tile appears with the wrist camera.
Click Layout in the top right of the episode view, then Plot. An empty plot tile appears that says "Choose fields to plot in the panel settings." The new plot tile is selected, so the left pane's third tab now reads Plot. If it shows a camera name instead, click the plot tile first.
Click the Plot tab. Expand /robot/derived and check gripper_state and ee_linear_speed.
Press play.

What you should see. Four tiles moving together. The gripper trace is a square wave: it sits at 0, steps to 1 when the gripper closes, and steps back down when it opens. In the first episode in the grid that happens just after 4 seconds into an 11-second recording, and it opens again about 1.3 seconds later. Drag the playhead, the vertical marker on the timeline at the bottom of the episode view, to that edge, and all three cameras land on the grasp at the same time. The end-effector speed trace dips as the gripper closes, which is the arm slowing down to grab, and picks up again as the arm moves off with the object. (If the camera tiles say "No data until 0:00.01," the playhead is at zero. Press play or nudge it forward and the video appears.)

Easy to miss. The Layout menu also has Saved layouts, Auto layout, and Reset layout. Feel free to save this four-tile arrangement and use it as your default. You can apply it to every episode you open. Reset layout gets you back to the two default cameras if you've added too much.

The Topics tab from step 3 also answers a data quality question: each camera lists 157 frames and each state topic 158 messages, all at about 14 Hz, a little under DROID's nominal 15 Hz, so the streams sit within one message of each other. A stream with dropped frames would list fewer frames here, and you'd see it now rather than after a training run. For a fuller walkthrough of checking MCAP recordings before training, see the MCAP robot log audit post.

The Voxel51 Add dataset page showing the Robotics Droid card, 1,000 DROID robot manipulation episodes as MCAP recordings, with an Add button.

A DROID episode in Voxel51 with two exterior cameras, the wrist camera, and a plot of gripper state and end-effector speed, with the playhead inside the span where the gripper is closed on a blue block.

If you'd like to read more, the MCAP glossary entry explains what's inside one of these files, and the multimodal data launch post covers why synchronized playback matters for end-to-end and VLA debugging.

Question 2: Which of these 1,000 demonstrations should I watch?

What we're doing. You're not going to watch 1,000 episodes, so the question is how to pick the 20 worth your time. The sidebar gives you three kinds of fields to pick with. Metadata is what was written down about each episode: which lab, which building, and which task. In the sidebar, lab and building sit in the Episode group and the task fields in the Task group, not in the group named Metadata. Signals are what the robot did, summarized into numbers and phase tags computed from the joint and gripper streams when this dataset was prepared. You'll find them in the Motion Stats group and under temporal tags, not in the group named Signals, which is empty here. Embeddings are what the cameras saw. An embedding is a list of numbers a model produces to describe what's in a clip, and clips that look alike get numbers that are close together, so plotting them puts similar scenes near each other. Two clips of the same drawer being opened from the same camera angle land next to each other, while a clip of the sink across the room lands somewhere else entirely. The interesting episodes are the ones where these disagree, and step 5 of the embeddings part shows you one kind to look for, where the metadata and what the cameras saw don't line up.

Where to click (metadata first).

Go back to the grid with the arrow at the top left. In the sidebar on the left, click the chevron on the Episode group to expand it. Fields are grouped (Tags, Primitives, Metadata, Episode, Task, and Motion Stats), and the chevron on a group header opens and closes it. The Signals, Events, and Summaries groups show 0 fields for this dataset, so you can skip them.
Check the box next to lab. The lab name appears on every thumbnail. Each field has three controls that do three different things: the checkbox puts the field's value on the thumbnails, the chevron opens a filter box, and clicking into that box drops down every value with its count. Click a value and the grid narrows to matching episodes. The value then shows as a chip under the box, with an X to remove it.
Click the chevron on lab, then click into its filter box. A list drops down with every lab and its episode count, smallest first, so scroll it to reach the big labs.
Now expand the Task group, click the chevron on task, and type drawer into its filter box. The suggestions are full sentences, mostly with one or two episodes each, because the field holds a free-text instruction a crowd worker wrote after the episode was recorded. DROID collected up to three per episode. Two related fields help. task_alt_1 holds a second instruction for 698 of the 1,000 episodes, and task_prompt holds the prompt DROID's collection app showed the operator, a coarse task family such as Move object into or out of container (227 episodes) or Move object to a new position and orientation (121).

What you should see. Twelve labs, and not an even split.

Lab	Episodes	Lab	Episodes
TRI	204	IPRL	80
AUTOLab	162	PennPAL	57
RAIL	116	ILIAD	41
IRIS	115	GuptaLab	25
CLVR	89	RPL	19
REAL	82	RAD	10

Episodes per contributing lab in the 1,000-episode DROID slice loaded in Voxel51. Two labs account for 37% of the slice.

Pick RAD from the lab list and the grid drops to ten episodes. That's the whole contribution from one lab, and it took three clicks to find. If you were about to fine-tune on this slice, you now know that whatever RAD's scenes look like, your policy will see very little of them.

To be fair to your notebook, the lab table itself is a one-line groupby if you have the metadata in a dataframe. What the click gets you is that the ten RAD episodes are now on your screen, you can open any of them, and in a moment the embeddings plot will show you where those ten sit. A dataframe tells you there are ten. The grid shows you what they are.

Some of the fields in the sidebar aren't in DROID's metadata at all. The Motion Stats group and the phase tags under temporal tags (idle, reaching, grasping, transporting, placing, releasing, and retreating) were computed from the recorded joint and gripper streams when this dataset was prepared for Voxel51, and the phase tracks you'll see on the timeline in Question 4 come from the same step. To get those into a dataframe you'd parse every MCAP file, resample the signals, and write the phase detection yourself. That's the part of the sidebar that isn't a groupby, and it's where Your turn below picks up.

Where to click (signals second).

Clear the lab filter first with the X on the count badge in the sidebar's search box. Then expand the Motion Stats group. Each field here is one number per episode computed from the robot's signals. peak_ee_linear_speed is the fastest the hand moved, and gripper_open_frac is the share of the episode the gripper spent open. Click the chevron on gripper_open_frac and drag the right end of the slider down to 0.25, or type 0.25 in the max box under it and press Enter. The grid drops to 37 episodes. Those are the episodes where the gripper was closed for at least three quarters of the recording, which usually means it was holding something.
Clear the gripper_open_frac filter with the same X. Then, in the Tags group, click the chevron on temporal tags and click into its filter box. It lists the seven phases each episode was cut into, with how many times each one occurs: idle, reaching, grasping, transporting, placing, releasing, and retreating. Pick idle. The grid narrows to 956 episodes, nearly all of them, so idle won't help you choose episodes. It tells you something about almost every episode instead: the arm spends stretches of each recording not moving. That's why Before you train on DROID filters idle timesteps rather than idle episodes.
Open one of them. The timeline under the cameras is already expanded, and each phase is drawn as its own track, so you can see where the idle stretches fall and jump straight to one. You'll meet this timeline again in Question 4.

What you should see. Two filters, a few seconds each, and you've searched every episode in the dataset for a physical behavior rather than a label. Nothing you just filtered on exists in DROID's own metadata.

Where to click (embeddings third).

Go back to the grid with the arrow at the top left, then clear the sidebar filters again with the X on the count badge in the sidebar's search box so the grid shows all 1,000 episodes. Then click the plus next to the Samples tab and choose Embeddings from the Curate section. The grid is itself a panel, the plus opens others, and the two now sit side by side.
Click the visualization named auto_segment_embeddings. It's marked 3D, and it's marked Ready because the embeddings come with the dataset, so there's nothing to compute. In all, 14,281 points load. Each point is one 5-second window from one camera stream. A typical episode gives a few windows per camera, and there are three cameras, which is why there are about 14 points per episode rather than one.
Click Color by. Out of the box it lists only None and sensor, the camera stream each window came from. Click Add fields at the bottom of the list, pick building and lab, and click Schedule. The run takes about a minute, and on the free plan it counts against the 30 hours of job runtime you get each month, though this one uses only seconds of it. When it finishes, click the refresh icon next to Add fields and both fields appear in the list. If you want more room, the fullscreen icon in the plot toolbar gives the plot the whole screen.
Change Color by to building. Scroll the legend and you'll find 38 values rather than a tidy list of rooms, including N/A, both AUTOLab and Autolab, and four spellings of Glen's office. The building field is whatever the collector typed, so normalize it before you group by it.
Now change Color by to lab and double-click RAD in the legend. Double-click isolates a value, and a single click hides it. Only RAD's 276 windows stay in the plot, about twice the per-episode average because RAD's episodes run long, and the grid beside it narrows to RAD's 10 episodes, because a selection in the plot filters the grid. The colored bars under each thumbnail mark its selected windows, one color per camera stream, and you'll use them again in Question 3. Look at where the windows sit, then double-click RAD again to bring the rest back and try a bigger lab such as TRI.

What you should see. Step 4 colors the cloud by location, and the legend lists each building with its point count. Isolate one of the smaller buildings and look at where its points sit. If they form their own region, the embedding model saw something consistent about that location, such as the table, the lighting, or the camera placement, and grouped its windows together. If they're spread through the cloud, the scenes at that location look like scenes everywhere else, which is what you'd hope for in a dataset built for diversity. One caution about reading the plot: it's a UMAP projection, a method that squashes the high-dimensional embeddings down to three dimensions you can look at. UMAP keeps neighbors close but doesn't preserve distances, so a region that looks separate can be an artifact of the projection. Use the plot to find candidates and the episode viewer to confirm them.

Step 5 is where you check for the disagreement this section promised. Isolate RAD again, then open the building filter in the sidebar, and it tells you how many rooms those windows came from. All 10 RAD episodes come from three rooms with Bayes in the name, and most of their windows sit in one part of the plot. Rooms that share a building name are expected to look alike. The case worth opening is a lab whose windows sit together across buildings with different names, because then the cameras are seeing something the building field doesn't record. Run the same check in the other direction by isolating one building in the legend and reading the lab counts in the sidebar. A building with episodes from more than one lab is the same disagreement seen from the other side.

Easy to miss. Adding a field to Color by is a one-time run per visualization, so lab and building stay in the list next time. A selection in the plot also adds a filter in the sidebar, so clear both when you're done. If the plot ever looks wrong, the curved arrow in its toolbar resets it and clears any lingering selection. The sidebar remembers which groups you expanded, per dataset, so it will look the way you left it next time.

Your turn. Combine two filters. In the Task group, set num_grasps to 2 or more, which leaves 228 episodes. A clean pick is one grasp, so a second grasp can mean the first one slipped and the operator tried again, though a task with two objects, such as stacking one block on another, needs two grasps on its own. Then add the lab filter. Which labs do the multi-grasp episodes come from, and what does the wrist camera show between the first grasp and the second?

Voxel51 sidebar with the lab filter open, listing DROID's contributing labs with episode counts, smallest first: RAD with 10, RPL with 19, GuptaLab with 25, ILIAD with 41, PennPAL with 57, and IPRL with 80.

Voxel51 grid of DROID episodes filtered to gripper_open_frac of 0.25 or below in the Motion Stats group, 37 of 1,000 episodes, episodes where the gripper stayed closed for most of the recording.

Segment-level embeddings of DROID in Voxel51 colored by lab, with RAD isolated in the legend so only its 276 windows remain, and the grid beside it narrowed to RAD's ten episodes.

If you'd like to read more, the post on multimodal embeddings for physical AI covers what these embeddings are computed on, and the VLA generalization and data diversity post makes the longer case for why scene and task diversity matters.

Question 3: Where inside the episode does the grasp happen?

What we're doing. An episode runs from a few seconds to about a minute. The grasp is about one second of that. If you embed the whole episode as a single vector, the one second that matters gets blended with the approach, the retreat, and a lot of empty tabletop, and it stops being searchable on its own. Segment-level embeddings split each episode into 5-second windows and embed each one separately, so you can search for the moment rather than the episode. The window length was chosen when the embeddings were computed. Shorter windows pin a moment down more precisely, longer ones carry more context, and five seconds is a reasonable middle for tabletop manipulation. Since every episode in this slice succeeded, what varies is how the grasp happened and when, and that's a question about windows rather than episodes.

Diagram of one 11.2-second DROID episode as three camera streams, ext1, ext2, and wrist, each cut into 5-second windows. The gripper-closed span from 4.1 to 5.4 seconds is shaded and crosses the 5-second cut, and the wrist camera's 0 to 5 second window is outlined in orange as the window a text search returns. — The first episode from Question 1, cut the way the segment embeddings see it: three camera streams, each split into 5-second windows, nine windows in all. Across the 1,000 episodes that comes to 14,281 windows, and each one gets its own vector. The grasp crosses the 5-second cut, so it lands in two windows, which is one cost of fixed windows.

Where to click.

Clear any sidebar filters left over from Question 2 with the X on the count badge in the sidebar's search box. Then, in the search bar across the top of the page, which is natural-language search over the whole dataset, type robot gripper closing on a blue block and press Enter. The grid filters to 21 episodes.
Expand the Episode group and open the lab filter. Fourteen of the 21 are from AUTOLab. You searched for an object and found a lab. Hold that thought, because the check in Your turn below sorts out how much of that pattern comes from AUTOLab's tasks and objects and how much from the search. If a scene can tell you which lab an episode came from, it can tell a model too, and a model that learns "blue block means this table" has learned something about the room rather than the task. Whether a given policy does that is a question this post can't answer, but it's a reason to evaluate on a lab the policy never trained on, and Before you train on DROID comes back to it.
Clear the top search with the X at the right of the search bar, so the plot shows every point again. Then open the Embeddings panel if it isn't open (the plus next to the Samples tab, then Embeddings, then auto_segment_embeddings). Click Search by text in the plot's top right corner, type the same query, leave Top K at 25, and click Search. Note that you click Search, you don't press Enter. If the query shows up in the suggestions under the box, clicking it runs the search right away.
Look at the grid. Each thumbnail now shows a frame from the camera stream that matched, usually the wrist camera, and a colored band on its progress bar showing where in the episode the match sits. Hover a thumbnail to see a caption naming the stream, such as match: /camera/wrist/image.
Hover one of the orange points in the plot. The tooltip shows the episode ID, the window (for example 0s-5s), and which camera stream matched. Click the point and the tooltip stays open with a frame from that window, so you can see what matched without leaving the plot. The X closes it.
In the tooltip, click Find similar. The plot highlights every window that looks like this one, across labs.
Change Streams from All to /camera/wrist/image. The cloud drops to only the wrist camera's windows. For a grasp query, that's the camera that matters.

What you should see. Twenty-five windows resolve to 21 episodes, so a few episodes matched twice. Only the matched windows light up in the plot. The colored band on each thumbnail shows you what segment-level embeddings buy you. On a long episode the match is a sliver of the bar and the rest is context. On a short one, a 5-second window fills most of the bar.

Both searches gave you the same 21 episodes, and both mark the matched window on each thumbnail. Search by text encodes your sentence into the same space as the video windows and returns the nearest ones, so you know exactly what it did, and it's the one to use when you want the windows in the plot and Find similar. The top bar runs the same kind of segment search, and the bar at the bottom of the grid shows it: after either search it reads Act on 25 segments across 21 episodes.

Easy to miss. Top K counts windows, not episodes. The suggestion list under Search by text can show queries from other datasets, so ignore any that aren't about DROID. The X at the right of the top search bar clears the results and returns you to the full dataset. If you isolated a legend entry in Question 2, double-click it again before you search, because Search by text only searches the points the plot is showing.

Your turn. Before you trust the lab finding, check the search. With the 21 results on screen, check the box next to task in the sidebar so each instruction shows on its thumbnail, read all 21, then open any episode whose instruction doesn't mention something blue and watch its matched window. Count how many of the 21 actually show the gripper closing on a blue block. If most do, the search is doing its job and the AUTOLab pattern is about the objects in its scenes. If many don't, the search is matching AUTOLab's table and lighting rather than the block, and the lab pattern is partly an artifact of the retriever. Either answer is useful.

When we ran it, 5 of the 21 instructions named a blue object as the thing to move, and only one, from TRI, asked for a blue block. Of AUTOLab's 14, two named a blue object and none a blue block. Watching the matched windows adds to that: about half of the 21 have something blue in view, often a cup, a rack, or a holder rather than the thing being grasped.

Then we swapped the color. A red block query returned 19 episodes, 11 from AUTOLab (58%), and a green block query returned 13, 6 from AUTOLab (46%). The blue query's share was 14 of 21, or 67%, and AUTOLab is 16% of the slice. Ten of the 19 red block episodes were also in the blue block results. The episode totals, 21, 19, and 13, are just how many distinct episodes the top 25 windows came from, so compare the AUTOLab shares rather than the totals.

Then we checked the base rate. We counted the episodes whose task field mentions a block: 34 of the 1,000, and 15 of them, 44%, are AUTOLab's. So part of the pattern is real, because AUTOLab recorded far more block tasks than its share of the slice. The blue and red searches came back above that 44%, and green landed on it. With 13 to 21 episodes per query, read that as a lean rather than a measurement. Only 8 of the 21 blue results have a task that mentions a block at all. Scroll AUTOLab's thumbnails and you'll see a likely reason, small colored blocks sitting on the table in episodes whose task is something else. We didn't count those, so treat it as a lead too. Our read is that the search found a gripper near something block-shaped and was weak on color, and that AUTOLab's scenes supplied much of the rest. The training lesson holds either way. In this slice, block tasks concentrate in one lab, so a policy can pick up a lab's objects along with its tasks.

Bar chart of AUTOLab's share of five sets in the 1,000-episode DROID slice: the whole slice, 16% (162 of 1,000), the 34 episodes whose task mentions a block, 44% (15 of 34), and the blue, red, and green block search results at 67% (14 of 21), 58% (11 of 19), and 46% (6 of 13). An orange dashed line marks the 44% block-task share. — AUTOLab's share of the results for three color versions of the same query, next to its 16% share of the 1,000-episode slice and its 44% share of the 34 episodes whose task mentions a block. Raw counts are on the right.

Voxel51 grid of DROID episodes returned by a text search for a gripper closing on a blue block, the first of 21 results, each thumbnail labeled with its lab and marked with a colored band at the matched 5-second window, beside the embeddings plot with the 25 matched windows lit orange.

Voxel51 embeddings plot of DROID segment windows with the 25 matches for a blue block text search lit orange, and a pinned tooltip on one match showing a frame preview, the episode ID, a 0 to 5 second window, the ext2 camera stream, and a Find similar button.

Open DROID in Voxel51

If you'd like to read more about how the windows are built and embedded, check out the segment-level embeddings post from the engineering team.

Question 4: How do I keep what I found?

What we're doing. Everything so far has been exploration. If you don't save what you found, you'll end up finding it again. Voxel51 has three ways to keep what you found: temporal tags on a window inside an episode, subsets of a filtered grid, and an export that takes the cohort out of the browser. We'll do all three, plus the saved layout that keeps your screen arrangement.

Where to click.

Open one of the episodes from Question 3. The timeline under the cameras is already expanded into tracks. Down the left are the phase tracks that came with the dataset, such as reaching, grasping, transporting, and releasing, with a bar for each stretch of that phase. In a dataset with object labels this is also where label tracks appear, showing when each labeled object is present across the episode. DROID has no object labels, so you won't see them here.
Click the tag icon in the transport bar, just right of the timestamps. The timeline label changes to "Select Range."
Drag across the window where the gripper is closed. An "Add temporal tag" dialog appears with Start and End to the hundredth of a second, a label field, and Accept.
Type a label, for example grasp, and click Accept. The dropdown above the label field also lets you add the range to an existing tag instead. Your tag appears as a new track pinned to the top of the timeline.
Close the episode with the arrow at the top left. In the sidebar under Tags, click the chevron on temporal tags and click into its filter box. Your tag is in the list with a count of 1, alongside the seven phase tags that came with the dataset. Pick it to filter the grid to tagged episodes.
With a filter or search applied, click Save as subset in the bar at the bottom of the grid, then New subset. Name it and click Save subset. The subset now appears in the dropdown next to the dataset name at the top left, which reads Episodes, and it stores segments as well as episodes, so a tagged window stays a window.
Open your subset first by picking it from the Episodes dropdown next to the dataset name. Then click More at the top right and choose Export dataset. On the Download tab, choose Filepaths and tags under Data, and click Download. You get a CSV with one line per episode: the episode's file path in Voxel51's storage, plus its sample tags if it has any. Temporal tags stay in Voxel51, so the file tells you which episodes are in your cohort, not where inside them. Each file name matches the episode's uuid field, and each episode's source field points at its folder in DROID's raw release, so the list is enough to find the same episodes in DROID. DROID's raw release is public at gs://gresearch/robotics/droid_raw/1.0.1/, organized by lab, then success or failure, then date.
Click Unsaved layout at the top right of the panels, then Save as new layout, so your grid-plus-Embeddings arrangement is one click away next time. The dataset already comes with one called embeddings_panel_and_grid if you'd rather start from that.

What you should see. A tagged interval pinned to the top of the timeline, a new entry under temporal tags in the sidebar, and a downloaded file. Subsets remember which episodes and windows you were looking at, layouts remember how your screen was set up, and the export is the step that gets the episode list out of the browser.

Easy to miss. The Start and End fields in the tag dialog have plus and minus buttons, so you can nudge the boundaries after you drag rather than trying to drag precisely.

If the cohort needs labels, Voxel51's annotation tools turn a subset into a labeling project with a review workflow, which is worth its own post.

A DROID episode in Voxel51 with three camera tiles and a gripper state plot, and an expanded timeline below with a grasp temporal tag pinned at the top above phase tracks such as reaching, grasping, and transporting, with the playhead at the start of the grasp, the gripper closed on a blue block.

Before you train on DROID: five changes to make

Here is how what you learned would change a training run, in the order you'd make the changes. The counts below belong to the 1,000-episode slice, so treat the slice as a rehearsal. On the full DROID dataset, rerun the lab counts before you choose what to hold out.

Evaluate on a lab you didn't train on. Start with what you saw in Questions 2 and 3: two labs make up 37% of this slice, and a lab's objects, table, and room can reveal which lab an episode came from. Now think about what a random train/test split does with that. It takes episodes from every lab and puts most of them in training and a few in evaluation, so nearly every evaluation episode comes from a lab, and usually a room and a set of objects, that the policy already saw during training. When the policy scores well, you can't tell how much of that score comes from completing the task and how much comes from recognizing a familiar room.

The fix is to hold out an entire lab: train on eleven of the slice's twelve labs and evaluate on the twelfth, so the evaluation scenes are new to the policy. The DROID paper didn't evaluate this way. It trained each policy on a small set of demonstrations collected in the evaluation scene, mixed with DROID, and tested robustness by adding distractors or swapping objects in that same scene. Holding out a lab asks a harder question, whether a policy works in a room it has never seen. The 2024 paper listed that as an open question, and newer work reports progress on it: Physical Intelligence describes its π0-FAST-DROID checkpoint as performing simple tabletop tasks zero-shot in new scenes on the DROID platform. One thing to watch: labs also did different tasks, so holding out a lab shifts the task mix as well as the scenes. Use the task buckets from the next paragraph to check what the held-out lab covers, and read any drop in performance with both shifts in mind. RAD only has ten episodes, which is too few to evaluate on, so a mid-sized lab such as CLVR (89 episodes) or REAL (82) is a better choice. If you have the compute, repeat the run holding out a different lab each time and average the results. One caution about what "evaluate" means here. Without a robot, evaluating on the held-out lab means measuring how well the policy predicts the recorded actions, and that's a proxy. The real test is running the policy on a DROID robot in a new scene, which is what the RoboArena benchmark does on diverse tasks and scenes in the real world.

Diagram comparing two ways to split the DROID slice. A random 80/20 split trains on 12 labs and 800 episodes and evaluates on the same 12 labs and 200 episodes, so every eval lab was already in training. Holding out one lab trains on 11 labs and 911 episodes and evaluates on CLVR's 89 episodes, outlined in orange, so the eval lab was never seen in training. — A random split puts every lab in both training and evaluation. Holding out a lab such as CLVR means the evaluation episodes come from a lab the policy has never seen.

Bucket the task strings before you stratify. Question 2 hinted that the task field holds too many distinct strings for the sidebar to list, because each one is an instruction a person wrote, and "close the drawer" and "Close the bottom drawer" count as two different tasks. If you stratify your split by that field as it stands, you'll be balancing many near-duplicates with only an episode or two each. Collapse the instructions into a few dozen verb and object groups first (such as open, close, pick up, put in, and wipe) and stratify on those. The task_prompt field gives you a head start, because it already sorts episodes into the task families DROID's collection app offered, though some prompts are compound, such as Do any two tasks consecutively. The DROID paper does something similar when it counts its 86 tasks, treating each distinct verb as one task.

Filter idle timesteps. Go back to the ee_linear_speed trace you plotted in Question 1. Stretches where the line sits at zero are moments when the arm isn't moving, and DROID has many of them, partly because of how the VR teleoperation interface works. The openpi repository from Physical Intelligence, which publishes an approximate open-source reproduction of the π0.5-DROID training pipeline, filters these idle timesteps by default and reports that filtering significantly improves policy performance. Its filter drops timesteps whose next chunk of actions would be mostly idle, so it works on moments inside an episode rather than on whole episodes. The idle phase tag from Question 2 marks these stretches for you, and 956 of the 1,000 episodes have at least one. Scrub through a few episodes from different labs with the idle track and the speed trace open to see how much of each one sits still before you decide how aggressive your own filter should be.

Decide what to do with the regrasp episodes. In the Your turn exercise in Question 2 you looked at num_grasps. Episodes with two or more grasps include demonstrations where the first grasp slipped and the operator tried again, and also tasks that need two grasps, such as stacking one block on another or a task_prompt of Do any two tasks consecutively. Read the instruction and watch the wrist camera to tell them apart. There are two reasonable choices. One is to filter them out, because a policy can learn the hesitation along with the recovery. The other is to keep them, because a recovery is exactly the behavior a policy needs when its own first grasp slips, and it's rare in a dataset where every episode ended in success. The right answer depends on what you want the policy to learn. Whichever way you go, decide it on purpose, and save the regrasp episodes as a subset so you can test both.

Export the cohort. Once you've chosen a held-out lab and saved your subsets, run the Filepaths and tags export from Question 4 on each subset. Each export is a CSV listing that subset's episodes. Temporal tags don't come with it, so keep the windows you care about in the subset in Voxel51. Build your split from that list rather than from the raw dataset, and the next person on your team can reproduce your split without redoing the exploration.

None of this is specific to DROID. The Import button at the top of the page takes your own MCAP recordings, along with images, video, point clouds, and common label formats. Synchronized playback, the sidebar, temporal tags, subsets, and Export work on them as soon as they load. The signal-derived fields in Question 2 were prepared for this dataset, so expect to compute your own for your data. If you have your own teleoperation logs, that's the natural next step. The free plan's limits apply: three datasets, up to 10,000 samples each, and 20 GB of storage.

What we covered

Here is what you can now do on any dataset in Voxel51, and what doing it told us about DROID.

Question	What answered it	What we learned about DROID
What does one demonstration look like?	Synchronized playback with camera tiles and a state plot	Three cameras and robot state at about 14 Hz, and in the first episode the gripper closes about 4 s into 11 s
Which episodes should I watch?	Sidebar filters on metadata, motion stats, and phase tags, plus the Embeddings panel	Two of twelve labs contribute 37% of the slice, and task is free text rather than a category
Where does the grasp happen?	Natural-language search, segment-level Search by text, and Find similar	25 windows resolve to 21 episodes, 14 from AUTOLab, the lab behind 15 of the slice's 34 block tasks
How do I keep it?	Temporal tags, subsets, and Export	A tagged window becomes a sidebar filter, a subset keeps it, and Export turns the cohort into an episode list

Four questions about the DROID dataset and the Voxel51 feature that answered each one.

Frequently asked questions

DROID is a robot manipulation dataset of 76,000 successful teleoperated demonstrations collected across 13 institutions on a shared Franka Panda platform, released in 2024 to study whether scene and task diversity improves policy generalization.

Yes, a 1,000-episode slice of DROID is free to explore in Voxel51's Explore library, open to anyone who signs up with a work or .edu email address. Add it to your workspace and there is nothing to install.

The DROID dataset is in Voxel51's Explore library as the Robotics Droid card. Follow the DROID link in this post, or click Explore in the top navigation of the Datasets home page and search droid, then click Add on the card. The free plan holds three datasets, so you may need to remove one first.

The DROID data in Voxel51 is in MCAP. The DROID authors distribute it in RLDS and as raw recordings, Hugging Face hosts a LeRobot conversion, and the Voxel51 copy converts each raw episode to a single MCAP file so the three cameras, robot state, and actions play back on one timeline.

A segment-level embedding is a vector computed on a short window of a recording, here 5 seconds of one camera stream, rather than on the whole episode. Searching over segments returns the moment that matches instead of the episode that contains it.

Yes, Voxel51 works with your own robot data: the Import button accepts MCAP recordings along with images, video, point clouds, and common label formats. Playback, tags, subsets, and Export work right away. The signal-derived fields and segment embeddings in this post were prepared for the DROID slice, so plan to compute your own for your data. The free plan holds three datasets of up to 10,000 samples each, and past its limits you'll be routed to the sales team.

Credits and further reading

DROID was co-led by Alexander Khazatsky and Karl Pertsch with a team across 13 institutions, and is described in the paper DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset (arXiv:2403.12945). The dataset lives at droid-dataset.github.io. On the platform side, Kacey Wang's post explains how the segment-level embeddings are built, and the multimodal data launch post and the LeRobot support post, both by Prerna Dhareshwar, go deeper on the features than this post does.

Next in the series

This is the first post in a series. Each one takes a public dataset, asks four questions a practitioner would ask before training on it, and uses Voxel51 to answer them. The questions change with the dataset, and the approach works the same way on your own data. The next post takes an autonomous driving dataset through the same four-question approach, and the questions will be about rare events and redundancy rather than scene diversity.

Open DROID in Voxel51

Explore DROID in Voxel51: a hands-on tour of the DROID robot manipulation dataset

Try Voxel51 for yourself

Key takeaways

Who this post is for

About the DROID dataset

Question 1: What does one demonstration in DROID look like?

Question 2: Which of these 1,000 demonstrations should I watch?

Question 3: Where inside the episode does the grasp happen?

Question 4: How do I keep what I found?

Before you train on DROID: five changes to make

What we covered

Frequently asked questions

Credits and further reading

Next in the series

Try Voxel51 for yourself

Related posts

Related posts

Explore DROID in Voxel51: a hands-on tour of the DROID robot manipulation dataset

Try Voxel51 for yourself

Key takeaways

Who this post is for

About the DROID dataset

Question 1: What does one demonstration in DROID look like?

Question 2: Which of these 1,000 demonstrations should I watch?

Question 3: Where inside the episode does the grasp happen?

Question 4: How do I keep what I found?

Before you train on DROID: five changes to make

What we covered

Frequently asked questions

What is the DROID dataset?

Is DROID free to explore in Voxel51?

Where is the DROID dataset in Voxel51?

What format is the DROID data in Voxel51?

What is a segment-level embedding?

Can I use Voxel51 with my own robot dataset?

Credits and further reading

Next in the series

Try Voxel51 for yourself

Related posts

Related posts