Every robotics team has the same drawer. Somewhere in it is a script called something like parse_bag_v3_final.py, written at 1 a.m. before a demo, that reads exactly the topics that one robot happened to send that one week. Nobody trusts it enough to reuse it, so the next dataset gets its own script, and the one after that gets another. Getting data off the robot and into a usable dataset ends up hand-rolled, every single time, and every import script gets thrown away the moment the demo ends.
This post is about not writing that script. We are going to hand a real, untouched MCAP recording, straight off a robot, to an AI coding agent, and watch it turn into a browsable, synchronized, multimodal FiftyOne dataset without us writing a single line of parsing code. Then we will open things up and show the exact same result built by hand with the FiftyOne SDK, so you leave with two ways in: one for quick exploration, one for a repeatable pipeline.
Key Takeaways
Agent skills let an AI coding agent import a raw MCAP recording into FiftyOne as a synchronized, multimodal dataset with no hand-written parsing code.
The fiftyone-dataset-import skill reads only an MCAP file's channel schemas to detect a multimodal recording, avoiding the cost of decoding every message up front.
FiftyOne's multimodal viewer plays point cloud, RGB, thermal, depth, and IMU data from a single MCAP file on one shared timeline.
The CAVERS dataset, recorded in a Spanish cave with millimeter-accurate motion capture ground truth, is a public CC BY 4.0 benchmark for testing MCAP workflows.
Everything the skill does is built on the open source FiftyOne SDK, so a chat prompt and a checked-in pipeline script produce the same dataset.
Part one: what is an agent skill, actually?
Strip away the buzzwords and a skill is a folder. Inside it is a markdown file that tells an AI coding agent, step by step, how an expert would do a specific job: which files to look at first, which command to run, which traps to avoid, and how to tell success from failure. The agent reads that folder the same way a new hire reads a training guide, and then it does the work itself, in your terminal, on your data.
Voxel51 packages exactly this kind of runbook for FiftyOne into an open source repository, installed with one line:
Installing the skill pack. Eighteen skills, discovered and offered up in one shot, no manual wiring.
The installer clones the repository, discovers every skill inside it, and lets you pick which ones your agent should have access to going forward. Once installed, the skills live alongside your project and your agent picks the right one automatically based on what you ask for.
From the project README: skills bridge natural language and FiftyOne's own operator library.
The one this post is about is called fiftyone-dataset-import. Its job is exactly the drawer full of throwaway scripts: point it at a folder of images, a COCO export, a Hugging Face dataset, a directory of point clouds, or a raw robotics sensor log, and it scans the data, figures out what it is actually looking at, and builds the FiftyOne dataset for you. The part we care about here is the last one. Inside that skill is a dedicated path for MCAP, the container format robotics and autonomous vehicle teams use to record everything a robot's sensors saw during a run, camera frames, LiDAR sweeps, IMU readings, coordinate transforms, all combined together on one shared clock.
One .mcap file is not a preview of a robot's sensors. It is all of them, combined and time-synced, in one self contained file.
Part two: a real MCAP file, end to end
A skill only proves itself on messy, real data, so we did not reach for a clean, made-up sample. We went looking for a dataset that would really test the multimodal viewer: real hardware, real darkness, real sensor gaps, the kind of place search and rescue robots actually work in, not a lab bench. That search ended underground, in a cave in southern Spain.
CAVERS: Multimodal SLAM Data from a Natural Karstic Cave with Ground Truth Motion Capture. Franchini, Rodríguez-Martínez, Martínez-Petersen, Pérez-del-Pulgar and Chiaberge, 2026.
CAVERS was recorded at Cueva de la Victoria in Malaga by teams from Universidad de Malaga and the Polytechnic of Turin. A rover and a handheld rig, both carrying a RealSense D435i RGB-D camera, an Optris PI640i thermal camera, and a Velodyne VLP-16 LiDAR, were walked and driven through pitch dark passages while an Optitrack motion capture system, set up inside the cave itself, recorded millimeter-accurate ground truth pose at 120 Hz. In total it is twenty four sequences and roughly 335 gigabytes, released under an open CC BY 4.0 license, and as far as we could find, nobody had written about it before. A cave has no GPS, no steady light, and walls that look the same in every direction, which is exactly why it is a real test and not a toy example.
Figure 1 from the paper: the sensor rig, the rover, and the motion capture markers, all inside the cave that will not give the robot a single free landmark.
The four CAVERS sequences used in this walkthrough, and what each MCAP file's channel schemas revealed before decoding a single message.
The four CAVERS sequences used in this walkthrough, and what each MCAP file's channel schemas revealed before decoding a single message.
Sequence
MCAP channels
Ground truth topic
Notes
rec_handheld_8
14
Not covered in post
49.6 s, 481 MB, smallest full recording, main walkthrough
loc_handheld_1
16
Yes
7,915 ground truth poses decoded and plotted
loc_diablo_5
16
Yes
Carries the Optitrack odometry topic
loc_handheld_5
14
No
Likely left motion capture coverage, near-total darkness
We picked the smallest full recording, rec_handheld_8, forty nine point six seconds and a 481 megabyte download, so that you could repeat this whole walkthrough in minutes rather than hours. The ask to the agent was one sentence, typed once, in plain language:
The entire prompt. Download, unzip, import, done, no format flags and no parser code.
That single sentence triggers the skill's MCAP path. It downloads the file, unzips it, and instead of trying to fully decode every message up front, it reads just the channel schemas, a light table of what topics exist and what kind of message each one carries. That is enough to know the file is multimodal without paying the cost of reading every byte of sensor data in Python first. Fourteen channels came back: a Velodyne point cloud, a RealSense color and depth stream, a thermal camera, an IMU running at about 185 hertz, static transforms, and a few vendor specific topics the viewer would correctly leave alone. The skill builds the FiftyOne dataset from that schema table, and one line confirms what just happened:
Then we opened it, and this is the payoff the whole pain point was about. Not a script that half worked and needed babysitting, but a synchronized, tiled viewer you can scrub through, built from a file the agent had never seen a minute earlier.
rec_handheld_8 open in FiftyOne. Point cloud top left, live decoded IMU message top right, RGB, thermal, and depth tiles along the bottom, one shared timeline underneath all of it.
Scrub that timeline and every tile updates together: the Velodyne sweep traces the cave wall, the thermal camera catches a warm silhouette in the dark, the depth tile shades by distance, and the raw IMU panel prints real orientation and acceleration numbers off the actual recording. We wired the empty Plot tile to the IMU's linear acceleration on all three axes, and the result is the single frame that sells the whole idea of multimodality better than any paragraph could.
Five different views of the exact same instant: 3D point cloud, a synced three axis acceleration plot, the raw decoded IMU message, thermal, and depth. Nothing here was written by hand.
At one point mid session the agent paused to describe its own next move before making it, which is worth including here exactly as it happened, because it is a small honest window into what working with a skill actually feels like day to day.
The agent narrating its own reasoning mid task, before configuring the plot tile above.
The route, not just the room
The CAVERS paper itself tests seven SLAM algorithms, the software robots use to map a space and track their own position in it at the same time, by plotting each one's estimated path against the Optitrack ground truth. One figure in the paper shows this for three sequences in particular: loc_handheld_1, loc_diablo_5, and loc_handheld_5. Seeing that figure raised an obvious question: could FiftyOne show that same route, live, inside the viewer we were already using?
Figure 5 from the paper. Each panel is one sequence's estimated route against ground truth, plus a reconstructed point cloud of the room the team called Sala del Dosel.
We downloaded all three, and the schema scan right away told us something the paper's figure does not spell out: only two of the three, loc_handheld_1 and loc_diablo_5, actually carry the Optitrack ground truth topic inside their MCAP file. loc_handheld_5 has fourteen channels instead of sixteen, with no spaceuma/optitrack/odom topic at all, most likely because that walk went outside the area the motion capture system could track. That is exactly what a capability flag on the sample, has_ground_truth_trajectory, is for: you find out from the data itself, instead of assuming every file gives you the same thing.
For the two sequences that did carry it, we decoded all seven thousand nine hundred and fifteen ground truth poses in loc_handheld_1 and plotted them ourselves. The shape matched the paper's own route almost exactly: two loops through the same passage, both starting and ending close to the same spot.
Our own decode of the millimeter accurate motion capture trajectory for two sequences, plotted top down. Circle marks the start, square marks the end.
A static plot answers the question once. We wanted the route to live inside FiftyOne itself, growing in sync with playback the same way the point cloud and the camera tiles already do. That meant writing a real nav_msgs/msg/Path channel, in the same format ROS 2 (the standard software framework most robots run on) would use for it, and adding it into a copy of the original MCAP file without touching a single one of the other thirty seven thousand messages already inside it. Once the sample pointed at this new file, FiftyOne's own 3D Labels panel picked up the new channel, and turning it on drew the ground truth path as a growing line, following the playhead exactly like every other tile.
The orange line is the real motion capture trajectory, authored as a native MCAP channel and rendered by FiftyOne's own 3D viewer, growing as playback advances.
One more thing turned up almost by accident while looking at the transform tree for these sequences: FiftyOne has a frame graph tool hiding behind the Transforms tab, and it flagged exactly the kind of thing a careful reviewer would want to catch. The motion capture frame and the robot's own sensor frames sit in two separate, disconnected groups, with no link recorded between them in this file.
The full coordinate frame tree for loc_handheld_1, health checked automatically. The motion capture frame sits in its own disconnected component, exactly as expected.
And loc_handheld_5, the one sequence without ground truth, is still worth a look on its own terms. It has nearly none of the ambient light the other two sequences occasionally caught, and the depth tile shows a genuine void where the cave simply swallows the sensor's range.
loc_handheld_5: a real rock formation lit only by an onboard torch, and a depth tile with an honest hole in it where the cave outran the sensor.
This whole walkthrough was screen recorded as it happened, agent chat on one side, the App scrubbing through synchronized playback on the other, which is the closest thing to proof that none of it needed a human hand-writing a parser in the middle.
A frame pulled from the session recording: rec_handheld_8 mid playback, every tile still in lockstep with the same timeline.
None of this, the trajectory, the transform graph, the honest gaps, was staged. It came out of pointing a general purpose skill at a truly hard dataset and following what the data actually said.
Part three: the same MCAP import, in code.
A chat prompt is a great way to explore a new recording, but a pipeline that runs every night needs code, not a conversation. Everything the skill did above sits on top of the open source FiftyOne SDK, and it is worth showing in full, because the point of this whole post is that you can import your own MCAP files either way: talk to an agent when you are exploring, or run the script below when you are building something you will run again and again.
Reading the channel schemas needs nothing but the lightweight, pure Python mcap package, unrelated to any ROS installation:
From there, turning schemas into simple, searchable flags, and the file into a FiftyOne sample, is a few dozen lines:
That is the entire import: no bag reader, no message decoding loop, no custom viewer. The filepath's extension alone tells FiftyOne this is a multimodal sample, and the tiled viewer, the schema aware tile routing, and the synchronized playback clock all come from FiftyOne itself, the same way whether a human typed the code or an agent generated it on the fly from a sentence in chat.
The capability flags are what make this work past one file. Once dozens or hundreds of recordings are in a dataset this way, a single line answers a question that used to mean opening every file by hand:
The skill and the SDK are not two competing products. They are two doors into the same building. One is faster for a first look at a new recording, the other is what you check into your pipeline.
Where this leaves you
The pain we started with was never really about MCAP specifically. It was about every team quietly re-solving the same problem, getting sensor data off a robot and into something you can actually look at, and throwing the solution away as soon as the demo was over. A skill turns that one-off script into something an agent already knows how to do, correctly, for your files, every time you ask. The SDK underneath it means nothing about that path is a black box: read the skill, read the code it is built on, and use whichever entry point matches the job in front of you.
If you want to reproduce any of this yourself, the CAVERS dataset is public, CC BY 4.0, hosted on Zenodo and mirrored at github.com/spaceuma/cavers, the paper is on arXiv (2604.15052), and the skill that did the importing lives in the open source voxel51/fiftyone-skills repository, installed the same one line way it was in Part one.
CAVERS dataset: Franchini, G., Rodríguez-Martínez, D., Martínez-Petersen, A., Pérez-del-Pulgar, C. J., and Chiaberge, M. CAVERS: Multimodal SLAM Data from a Natural Karstic Cave with Ground Truth Motion Capture. Polytechnic of Turin and Universidad de Malaga, 2026. arXiv:2604.15052. Data licensed CC BY 4.0.
FAQ
There are two ways: ask an AI coding agent equipped with the fiftyone-dataset-import skill in plain language, or use the FiftyOne SDK with the pure Python mcap package. Both read the file's channel schemas and produce the same multimodal dataset with synchronized playback.
An agent skill is a folder containing a markdown file that tells an AI coding agent, step by step, how an expert would do a specific job. The agent reads it like a training guide and does the work itself in your terminal on your data.
No. Reading MCAP channel schemas requires only the lightweight, pure Python mcap package, which is unrelated to any ROS installation.
The filepath's extension alone tells FiftyOne the sample is multimodal, and dataset.media_type returns "multimodal" after import. Capability flags like has_pointcloud and has_imu come from the file's channel schemas.
CAVERS is a multimodal SLAM dataset recorded at Cueva de la Victoria in Malaga, with RGB-D, thermal, and LiDAR sensors plus motion capture ground truth at 120 Hz. It spans twenty four sequences and roughly 335 GB under a CC BY 4.0 license.
