The new
Segments.ai plugin for FiftyOne brings seamless integration between data curation and annotation, enabling ML teams to move from dataset exploration to high-quality data labeling in a single workflow. Built by Segments.ai ML Engineer Tom Roussel, this data labeling plugin connects FiftyOne’s powerful dataset inspection and error analysis tools with Segments.ai’s multi-sensor labeling platform. In this interview, Tom shares how the integration works, the benefits for annotation and quality control, and how teams can get started.
Tom: Our solution is a
multi-sensor data labeling focused tool, specifically. Our customers are typically building a
vehicle or
robot with multiple camera, lidar, and radar sensors. With a multi-sensor data labeling tool, customers can annotate different sensors in a single interface, getting consistent labels across all sensors. The main benefits include:
- Label multiple sensors simultaneously
- Visualize multiple sensors together
- Get consistent IDs across modalities and time
- Save time on quality control and edits
Here’s an overview of our products:
- Data labeling tool: We provide a multi-sensor data labeling tool, ready for use by internal, external, or computer vision engineering teams
- Data labeling services: We can help you find the right workforce, onboard them quickly, and set up your model-assisted labeling and active learning pipelines
- Data labeling advisory: We bring our Ph.D. expertise along to help customers improve their labeling specifications, pipelines, and strategies
How is this plugin bridging data curation and annotation?
Tom: Because
data curation and annotation go hand in hand, it’s natural that we want to reduce the friction between these two critical ML tasks. The
Segments.ai plugin allows the user to use the powerful set of data curation capabilities FiftyOne provides and easily use our extensive data annotation capabilities to iteratively improve the
quality of their datasets. The best part is the data labeling integration is completely seamless. This has enabled customers to request annotations directly within the FiftyOne interface and eliminating the hassle of switching between tools.
What are the benefits of using Segments.ai?
Tom: This multi-sensor data labeling plugin gives customers the ability to connect data exploration, annotation, and quality control in one smooth process. This reduces friction and dataset errors, leading to better datasets and faster iteration cycles.
These are some top benefits that Segments.ai plugin offers for customers:
- A seamless way to integrate data curation capabilities in our customer’s data annotation workflow
- Less friction between finding interesting/hard cases for models and having them annotated
- Less context switching when curating a dataset and communicating with the annotation team
- No separate scripts are needed for Segments.ai, as it’s a one-click integration
How does a multi-sensory data labeling tool work?
Tom: It is actually quite easy.
After installing the
segments.ai plugin, you start by uploading a dataset from FiftyOne to the Segments.ai annotation service. You can then either create a new “dataset” on our platform or append the data to an existing one. Afterwards, you have the option to interact with the Segments.ai dataset in a few different ways. You can create a persistent link between a FiftyOne dataset and a Segments.ai dataset. This is a little different from the way FiftyOne integrates with other annotator backends, where the dataset concept in the annotation backend may not be as long-lived. You can then fetch annotations made on the Segments.ai platform.
Lastly, our platform supports “issues” linked to samples in datasets. This is a mechanism to communicate errors, edge cases, etc., with the annotation team. Our plugin allows the user to create issues from within FiftyOne by selecting a sample and entering a description of the issue.
What led you to develop a multi-sensor data labelling tool?
Tom: We had multiple customers who contacted us asking if we had an integration with FiftyOne. After reviewing our platform, we agreed that it would be a helpful integration, because of FiftyOne’s plugin system it gave us an easy way to develop the integration.
Accelerating high-quality multi-sensor datasets
In summary, the Segments.ai plugin for FiftyOne offers ML teams a streamlined way to explore multi-sensor data, surface issues, and request annotations— without switching tools. This integration reduces friction, improves annotation quality, and accelerates iterative development for visual AI teams. To start using the plugin, explore the Segments.ai resources linked below, review the GitHub repo, or try the workflow directly in FiftyOne.
Plugin resources
- This blog includes helpful tutorial videos on how to get started
- When you’re ready to install and configure the plugin, head on over to the GitHub repo
- If you’d like to see the plugin in action, we’d love to show you—reach out for a demo
