Frequently Asked Questions

Question 1

Does FiftyOne support multimodal data?

Accepted Answer

Yes, FiftyOne supports multimodal data. FiftyOne is a unified data platform for physical AI that brings visualization, curation, and annotation of multimodal, time-series data into one place. Teams inspect synchronized camera, LiDAR, sensor, and robot state data, then search and curate it at scale to build better datasets for autonomous vehicles and robotics.

Question 2

What multimodal data formats does FiftyOne support?

Accepted Answer

FiftyOne natively ingests MCAP, the default recording format for ROS 2, and works with data converted from rosbag. You can also bring your own extracted frames and post-processed sensor data.

Question 3

Which sensor modalities can I inspect together?

Accepted Answer

Camera streams, LiDAR point clouds, and numeric time-series sensors like IMU, acceleration, and CAN bus signals, along with robot state, all play back in sync on a single shared timeline. Every modality stays aligned, so you can inspect a complete episode in context rather than one sensor at a time.

Question 4

Do I need to extract frames from my video first?

Accepted Answer

No. FiftyOne Multimodal works directly with encoded stream media and infers frame-level data from it, so you do not have to pre-extract frames before ingesting. If your pipeline already produces extracted, post-processed frames, you can bring those too.

Question 5

How is this different from a visualization or observability tool?

Accepted Answer

Visualization tools make it easy to inspect a single robot episode or failure. FiftyOne Multimodal adds what those tools do not: large-scale curation, embeddings and semantic search, and annotation, so you can go from finding an edge case to a labeled, model-ready dataset in one platform.

Question 6

How do I find edge cases and failure modes across a large dataset?

Accepted Answer

You can query by metadata, temporal events, and annotation labels, then search by visual and semantic similarity using embeddings. This surfaces rare scenarios, recurring failure modes, and underrepresented data across an entire dataset, the same longtail mining workflow autonomous vehicle teams rely on.

Question 7

Can I tag and annotate multimodal data?

Accepted Answer

Yes. You can tag arbitrary time intervals to curate segments for investigation, work with 2D bounding box and 3D cuboid labels at the message level, and view labels as tracks over time. Label tracks show exactly when an object appears or an event occurs across a full scene.

Question 8

How large a dataset can FiftyOne handle?

Accepted Answer

It is built for datasets ranging from thousands of scenes to hundreds of millions of frames, with support for long, multi-hour encoded video and an open-ended number of time-series sensors. Purpose-built indexing keeps search, filtering, and playback fast at that scale.

FiftyOne Multimodal

E2E data platform

Turn physical AI development into a data flywheel

VLAs for Robotics & AV

Multimodal data curation across the VLA lifecycle

Improve VLA generalization with less data

Search petabyte-scale data for the longtail

Search & Filter

Query multimodal data at scale

Visualization

Visualize MCAP data with synchronized playback

Production Ready

Enterprise-grade physical AI data platform

Get started with FiftyOne Enterprise

Questions?
We have answers.