Announcing FiftyOne 1.0: A Major Milestone to Making Visual AI a Reality

I’m thrilled to announce the general availability of version 1.0 of FiftyOne open source, a major milestone on our journey to provide the open source foundation for building highly performant visual AI systems.

Why 1.0, and why now? FiftyOne has reached the robustness, flexibility, and value creation needed for real-world use across the diversity of scenarios in visual AI:

• FiftyOne has over 2M installs, 8K GitHub stars, has been thoughtfully crafted by over 100 contributors, and is trusted by hundreds of thousands of AI builders.
• Dozens of Fortune 500 companies across industries from automotive to security, healthcare, agriculture, robotics, and more, trust FiftyOne Teams as an essential part of their AI development.
• FiftyOne provides the comprehensive integrations, extensibility, and APIs required to be successfully deployed in real-world environments, including a powerful plugin framework and vector search integrations.
• FiftyOne natively supports a broad range of data formats, types, and sources used in visual AI projects, including the recent addition of 3D scenes.

With this release, we’re also excited to reaffirm our commitment to full stack open source AI. More on that later.

What’s the secret to this adoption? FiftyOne’s core goal is to accelerate and automate the visual AI development process by giving AI builders access to powerful, transparent, and open tooling that enables the development of better models powered by better data. FiftyOne uniquely brings data and model intelligence into a unified view, enabling you to explore, visualize, and curate higher quality datasets and iterate on your models more quickly, all while integrating flexibly and securely into the way you work. In short, FiftyOne enables visual AI projects to succeed. Moreover, FiftyOne Teams is built around FiftyOne Open Source at its core, benefits from its continued advancement, and unlocks the value of FiftyOne securely and at scale for enterprises.

But first and foremost, we are here today because of the unwavering support of our community, so I’m excited to share their enthusiasm around this milestone too! 

“FiftyOne has greatly enhanced our ability to understand data, helping us uncover and resolve challenges with our models, and achieve our project goals. Reaching the 1.0 milestone is an epic moment, reflecting how far the community has advanced and rallied around this product.”

—Adonai Vera, Machine Learning Engineer at Subterra AI and former co-founder and CTO of SwitchAI 

“It’s exciting to see FiftyOne evolve into such a powerful solution and with this 1.0 milestone release, the tool is hitting its stride – mature, flexible, extensible, and the community… one of the best I’ve seen! The tool has redefined how I explore and interact with datasets, offering advanced features like filtering, embedding, clustering, and seamless integration to give me those deep insights in a way that’s just effortless. FiftyOne has become the central hub for all my data needs.” 

—Joy Timmermans, Machine Learning Engineer at Secury360 and active FiftyOne contributor

Want to dive straight into the new capabilities in FiftyOne 1.0? Learn more in the What’s New section, or continue reading on.

Bridging the Gap Between Data and Models

The development of FiftyOne and its adoption to date have been driven by the need to bridge a divide between data and models that has existed in AI tooling.

Many solutions in the market today specialize in specific aspects of data management and model development like data labeling, data cleaning, model testing, experiment tracking, and monitoring. However, these tools lack the ability to deeply connect data and models.

Why does this deep connection matter? Data quality is the fundamental determinant of a model’s performance. AI teams need complete transparency into the source data on which they’re building and the ability to get hands-on with the data while training and evaluating their models. Without this mindset, a majority of AI projects simply fail.

Many teams attempt to fill this need by investing substantial resources into building complex in-house solutions that are inefficient, expensive, hard to maintain, and harder to scale; they waste effort manually curating datasets and performing ad hoc model analysis; and worst of all, they pour ever-increasing resources into model training without transparency into the fundamental limitations of their underlying data.

FiftyOne bridges the data-model gap by providing:

• Data management and visualization: FiftyOne allows you to perform deep data analysis by visualizing patterns and clusters or isolating critical scenarios to improve model performance. No more guessing which samples (or lack thereof) are degrading performance.
• Powerful workflows: FiftyOne provides a growing library of builtin workflows that make it easy to bring your data into your AI work, including rich model evaluation with measures such as quality, accuracy, false positives, and uniqueness at the sample level to iteratively tune and build high-quality datasets. It also allows you to interactively explore the strengths and weaknesses of your models alongside their source data to take actions that meaningfully improve performance. No more ad hoc data wrangling and siloed model analysis.
• Rapid iteration: custom/in-house solutions are hard and expensive to manage and scale. FiftyOne fits seamlessly into the stack that AI builders most often use (datasets, annotation tools, data pipelines, models, training, deployment) and provides a wealth of out-of-the-box visualization and analysis capabilities that bring the best practices of data-centric model development to your projects. No more siloed and inconsistent development practices across your team.
• Flexibility and extensibility: we optimize for the core tenets of adoption that AI builders seek: ease of use, interoperability with their existing AI stacks, extensibility to satisfy growing needs, and security and transparency to meet governance requirements. No more inflexible tooling that requires you to adapt your development to how it works and fails to support your needs beyond the proof-of-concept phase.

Our Commitment to Open Source

At Voxel51, we’re long-standing believers in the virtues of full stack open source AI, from model architectures and data sources down to the core infrastructure on which these datasets and models are built and refined. That’s why we’re proud that the core FiftyOne project is and will continue to be open source and permissively licensed.

Demonstrating this commitment, as part of the FiftyOne 1.0 release I’m thrilled to announce that FiftyOne Brain, the core component of the FiftyOne ecosystem that implements the AI/ML capabilities that enable users to automatically analyze and manipulate their datasets and models, is now fully open source!

Over thousands of conversations with everyone from scientists and researchers in academia to engineers and executives setting the vision for leading AI-forward enterprises, we’ve distilled some guiding principles that inform how our commitment to open source is critical to making visual AI a reality:

• Collaboration and innovation: open source AI democratizes access to powerful tools and resources, enabling researchers, developers, and startups to collaborate and build upon each other’s work. This accelerates innovation, lowers barriers to entry, and makes the evolving best practices of AI accessible to everyone.
• Transparency and trust: making the underlying code and data behind AI projects openly available promotes transparency and mitigates the risks of hidden biases, ethical concerns, and opaque performance by making everything publicly available for review. This mental model is becoming better understood for AI models, but it’s critically important for the data on which these models are trained and the platforms used to refine them as well. In contrast, proprietary AI software obscures how data and labels are generated, decisions are made, and how models operate.
• Customization and adaptability: providing open access to data frameworks and analysis tools enables AI builders to standardize around best practices and accelerates the overall progress of the field, including the cross-pollination of best practices across industries and domains that might not otherwise have the resources to invest in vertical-specific solutions.
• No vendor lock-in: open sourcing the AI stack helps avoid over-reliance on proprietary models and platforms that can trap organizations into ecosystems that lack long-term alignment with their evolving vision for AI. Adopting open systems provides both individuals and enterprises full control over their AI development initiatives.

You can learn more about FiftyOne Brain and how open source AI is unlocking technical breakthroughs in this blog by Jason Corso, my co-founder and Chief Science Officer.

And if you have ideas to extend FiftyOne Brain’s capabilities and want to add value to the community, contributions are welcome. We’ll celebrate your work!

What’s New in FiftyOne 1.0

With this major release, we’re advancing FiftyOne’s mission by expanding its commitment to extensibility and flexibility–which users and customers have consistently told us is an absolutely critical requirement of their AI development platform–providing even more ways to customize your FiftyOne environment to suit the unique requirements of your domain and surface key insights into the integrity of your data and models.

Click into the headings below to deep dive into some key features that version 1.0 of FiftyOne brings.

• Open source ML techniques: FiftyOne Brain, the core component of the FiftyOne ecosystem that implements the AI/ML capabilities that enable users to automatically analyze and manipulate their datasets and models, is now fully open source!
• Modal panels: we’ve introduced the ability to develop and use “micro” plugins that explore individual data samples and scenes, which complements FiftyOne’s existing support for custom “macro” plugins that explore entire datasets.
• Change tracking: all FiftyOne datasets now have automatically populated creation/modification fields on their samples, enhancing the capability of FiftyOne as a system of record for your visual AI data.
• Remote zoo models: FiftyOne natively supports downloading and inferencing with custom models whose definitions are stored in GitHub, complementing the hundreds of natively available models and existing integrations with popular frameworks like Hugging Face Hub and Ultralytics.
• Remote zoo datasets: FiftyOne also now natively supports loading datasets whose download/preparation instructions are stored in GitHub, completing the dozens of natively available datasets and existing integrations with popular data standards.

Many of these features establish interfaces that we’ll be making heavy use of over the coming months to bring new native experiences for loading, searching, tagging, and refining your visual data in FiftyOne. So stay tuned!
As always, you can give the latest release a test drive by installing FiftyOne locally:

pip install --upgrade fiftyone

Check out the release notes for a full rundown of additional enhancements and bug fixes in FiftyOne 1.0. 

Remember that all of the features above are also supported in FiftyOne Teams, where they can be used collaboratively, securely, and at enterprise scale.

Our Vision for FiftyOne

Every day, we learn of new real-life use cases that FiftyOne helps solve. In turn, our growing team of ML and data infrastructure experts are busy researching and developing innovative new features and workflows to continue bridging the gap between data and models.

FiftyOne’s roadmap over the coming months includes:

• Multimodal AI: we’ll expand our commitment to multimodal AI by adding support for new data modalities and more functionality to our suite of existing modalities (image, video, point clouds, 3D meshes, geolocation, documents, embeddings, and more); unlocking new use cases, verticals, and more powerful data/model analysis.
• App performance: as AI hardware and models continue to mature, the appetite for ever larger datasets will grow across use cases and industries. We’ll continue to invest in FiftyOne’s core infrastructure to ensure that we meet and exceed the pace of this growth by supporting larger data volumes and more complex data representations and queries.
• ML-powered workflows: we’ll advance FiftyOne’s mission to serve the full lifecycle of AI development by introducing new panels that use ML to automatically analyze and improve dataset and model quality.
• Ecosystem integrations: FiftyOne is designed to integrate naturally with other key parts of the AI stack. We’ll continue to deepen FiftyOne’s integrations through exciting new collaborations with additional AI components such as data lakes and workflow orchestrators.

In addition to our product roadmap, we are also investing in education. At Voxel51, we’re passionate about helping people build the best AI possible. So we’re pouring time and care into crafting learning resources that enrich the community: tutorials, workshops, course materials, office hours, and in-person interactions through meetups and events. Come say hi!

Key Recent Features 

In case you missed it, here are some recent features that are now generally available in FiftyOne 1.0:

Panels

Panels provide a powerful framework for building custom App panels via simple Python or JS interfaces that include a wealth of builtin components to convey information, create tutorials, show interactive graphs, trigger operations, and generally fine-tune your FiftyOne environment to surface the insights you need to develop high quality datasets and models. Think of Panels as miniature full-featured data applications that you can open in App spaces and interactively manipulate to explore your dataset and update/respond to updates from other spaces that are currently open in the App. 

FiftyOne 1.0 includes support for Python Panels, enabling ML engineers who are most comfortable in Python to build the kind of interactive data applications that previously required frontend expertise.

What can you build with Python Panels? The Dashboard Panel is the first generally available example, but the possibilities are endless, and we’ll be using the panels framework ourselves over the next few months to roll out a variety of new in-App experiences for workflows like model evaluation, identifying data quality issues, auto-tagging, and much more! 🚀

Vector Search Integrations

FiftyOne has rich integrations with many popular vector databases, allowing you to perform natural language or image searches on your FiftyOne datasets at the billion sample scale:

Pinecone | Qdrant | Milvus | MongoDB | Elasticsearch | Redis | LanceDB

To enable natural language and image search on your datasets, simply follow these instructions to add multimodal embeddings to your dataset:

import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")

# Index images
fob.compute_similarity(
    dataset,
    model="clip-vit-base32-torch",
    brain_key="img_sim",
)

session = fo.launch_app(dataset)

Whenever your dataset has one or more similarity indexes that contain multimodal embeddings, you can search your current dataset or view by typing anything into the text prompt above the App’s grid:

Check out the docs for complete information on performing text and image searches on your FiftyOne datasets at scale.

3D Scenes

FiftyOne 1.0 includes full support for 3D scenes, including meshes (GLTF, OBJ, PLY, STL, FBX), point clouds, and arbitrary 3D geometries composed of boxes, spheres, cylinders, and planes:

Community Updates

Shoutout to the following community members who contributed to FiftyOne 1.0:

• Evatt Harvey-Salinger contributed #4733 Added Med-SAM2 to the Model Zoo
• Kemal Eren contributed #4826 Fixed an off-by-one error when converting semantic segmentations to/from instance segmentations

What’s Next?

• If you like what you see on GitHub, give the project a star.
• Get started! We’ve made it easy to get up and running in a few minutes.
• Join the FiftyOne Slack community, where you can always find people happy to help.