Success story at a glance
Company:
- Name: RIOS Intelligent Machines
- Industry: Robotics
- Data capture: Multiple cameras in robot environments, including 360-degree camera views of the scene in which the robot operates
- Data sources: Robots from customer sites, headquarters, and synthetic data
- Data types: Images, videos, and 3D data
Challenge:
- RIOS supports a variety of robotic solutions deployed across a number of different customer environments
- RIOS needed a dataset management solution to efficiently organize and visualize its datasets, with the flexibility to plug into other components in the ML pipeline
- As an end-to-end automation company, RIOS wanted to eliminate unnecessary manual work and automate as much of the data engineering pipeline as possible
Solution:
- FiftyOne Teams helps RIOS efficiently organize and analyze 20TB+ of data, like GitHub for code, but for datasets
- FiftyOne Teams seamlessly plugs into RIOS’s ML experiment tracker, cloud data lake, and annotation tools, enabling a slew of automated ML workflows
- Data automation eliminates repetitive data transformations, freeing up valuable engineering time
- Continuous model improvements support RIOS’s 99.6%+ uptime guarantee
Introduction
RIOS Intelligent Machines is a game changer in the automation industry, helping enterprises automate their factories, warehouses, and supply chain operations while increasing production and eliminating defects.
RIOS accomplishes this through robust, reliable, and flexible AI-powered robotic solutions that seamlessly adapt to production requirement changes and AI-powered vision for monitoring operations, quality assurance, and handling complex operations previously not possible with mechanical robotic automation.
Challenge
Training different machine learning models on data from robots across customer environments, such as packaged food products, beverage distribution, and wood products, requires subtle changes in the data schema between each use case. If data transformations are done manually, a significant amount of data engineering effort is spent on turning raw data into usable data and then exporting it into a format compatible with model training.
Having a central dashboard to organize and visualize datasets and search and sort them would free up valuable engineering time to continue building the robotics and AI solutions that RIOS customers have come to know and love.
Solution
Complex manipulation tasks require 3D scene understanding, and RIOS’s ML team required a dataset management solution with a flexible metadata structure to add custom fields such as camera pose, object pose, and other 3D metadata. Equally important was the ability to export to common data types like YOLO and COCO for downstream model training tasks. Additionally, because RIOS already had an ML experiment tracking tool, annotation tools, and cloud data storage, it was critical to have a data management tool to plug into and supplement them.
Enter FiftyOne Teams.
Not yet familiar with FiftyOne Teams? It’s where real AI work happens. FiftyOne Teams helps you visualize, augment, manage, and QA data; it also helps you streamline the workflows that make enterprise machine learning possible. FiftyOne Teams extends open-source FiftyOne with added features enterprises need, including dataset permissions, versioning, sharing, support, and more.
The hub that connects the MLOps spokes
RIOS selected FiftyOne Teams as its central hub for datasets and integrated its other MLOps activities – annotation, ML experiment tracking, and synthetic data generation – into the FiftyOne platform.
RIOS’s data comes in from various sources, including data collected from robots at headquarters, data from robots in the field, and synthetic data, and is put into a cloud data lake. Once datasets are collected, they are processed and loaded into FiftyOne. From there, ML engineers can kick off processes for training, model analysis, annotations, simulations, customer reviews, or anything they need.
“Everything we do from the MLOps side interacts with FiftyOne Teams. It’s becoming the hub where all the spokes are connected. It’s like GitHub for code, but for our datasets,” said Matt Shaffer, VP of Artificial Intelligence and Co-founder at RIOS.
Automation and efficiency across the board
RIOS is always looking to automate as much as possible, including its own internal ML pipelines. RIOS’s ML team has built processes that understand that data as it’s flowing into the data lake and automatically put it into the FiftyOne format so it’s immediately ready to work with.
“The FiftyOne Teams API is clean and clear, making it easy to add fields and create samples in grouped datasets. One example is when I created a custom script to load in and render an annotated synthetic video dataset, which was very quick because of FiftyOne,” said Joshua Zorn, Senior Synthetic Data Developer at RIOS.
In addition, the ML team makes heavy use of FiftyOne’s tagging feature for efficiency. Being able to tag data makes it easy to find the exact data you’re looking for. RIOS tags data by customer, by object, and annotation status so that they have total transparency and visibility into which customer the dataset belongs to, the objects in the dataset, and what stage it’s in in terms of processing.
Easy scene reconstructions with grouped datasets
Working with robots often involves working with data from a 360-degree imaging system, with images from all four sides of an item and sometimes the top and bottom. With FiftyOne, RIOS can group all of those samples together to visualize a reconstruction of the scene. Using the grouped dataset feature of FiftyOne is now a common way for the RIOS team to interact with and view their data.
Adding real value to synthetic data
RIOS uses synthetic data, digital twins, and simulations to rapidly innovate and test new solutions for customer environments while sidestepping some of the constraints around prototyping in the real world, such as shipping times. FiftyOne Teams sits at the heart of RIOS’s synthetic data generation and virtual work cell pipeline.
“We use FiftyOne Teams for our dataset management system, which allows us to visualize our synthetic data, organize it, preview it, and ensure that we can work efficiently with it,” explained Joshua.
Continuously train and deploy models that perform
RIOS relies on FiftyOne Teams to visualize, analyze, and evaluate how their models perform on training data and how they perform on test data from production environments to gain critical insights from training through production.
“On one side, we use FiftyOne Teams during model training to prep and select the data, then inspect the model’s performance on that data. On the other side, we take the inference results from our production models and test data to visualize and evaluate them in FiftyOne Teams. Doing this means we have an end-to-end process that closes the entire loop for us on model analysis,” explained Shubham Kanitkar, Sr. Robotics ML Engineer at RIOS.
“Loading the inference results from production models back into FiftyOne Teams enables anyone on the ML team to review them to see whether or not they thought the model was performing well. Because of the rich visualizations provided by FiftyOne teams, it’s possible to review the samples quickly and efficiently for insights,” said Matt.
Collaborating with outside annotators
FiftyOne Teams’ roles and permissions set the stage for easy, streamlined collaboration with annotators from the customer’s side.
“In the past, we’ve collaborated with customers through self-hosted annotation platforms such as CVAT. This made it difficult to collaborate with outside organizations due to VPN policies and data privacy issues. Using FiftyOne Teams as a customer-facing app makes it easier for us to share data with customers to show them the quality of that annotation run,” described Shubham.
Unparalleled support & documentation
RIOS appreciates the quality of support that comes with FiftyOne Teams.
“I think of Voxel51’s customer success team as an extension of our team. We can work together on solving real-world ML problems and partner in cases where we’re requesting new features in the FiftyOne platform. It’s been a good working relationship for us,” explained Matt.
The RIOS team finds the documentation useful, too.
“Usually documentation goes unnoticed, but with FiftyOne, the documentation is killer. Having really good documentation is very important. Knowing that FiftyOne is well documented is much appreciated,” explained Shubham.
Results
- A growing data lake of 20TB+ of data supporting millions of customer objects handled
- Elimination of repetitive data transformations, freeing up valuable engineering time
- Continuously improve model performance to support a 99.6%+ uptime guarantee for customers
- Greater confidence in the quality of their datasets
