FiftyOne Enterprise 2.19.0 introduces the FiftyOne Agent, major upgrades to in-app annotation, a new Observability page, and much more! This release includes everything from the open source FiftyOne 1.16.0 release, plus Enterprise-exclusive features, along with dozens of bug fixes and feature improvements across FiftyOne Enterprise and FiftyOne Open Source.

Here’s what’s new:

At a glance

🤖 The FiftyOne Agent — work with your data in natural language, powered by 100+ LLM providers

✏️ Annotation Ontologies & click-to-segment — versioned label schemas and SAM2-assisted masks

📊 Observability — live metrics, logs, and telemetry for every delegated run

🛠️ Quality-of-life upgrades — SAM3 in the Model Zoo, private model-weight sourcing, new operators

🛡️ Security & resilience — CVE fixes, CAS hardening, and dependency upgrades

🤖 The FiftyOne Agent: Work with your visual data using natural language

Working with large visual datasets has traditionally meant constant context-switching between the App, Python SDK, notebooks, and docs just to import some data, run a model, or hunt down duplicate samples. In FiftyOne Enterprise 2.19.0, you can simply ask in the App.

The FiftyOne Agent is an AI-powered assistant that lets you work with your datasets using natural language — including data import, model inference, duplicate detection, model evaluation, and annotation — all from a single conversational interface.

Examples of tasks you could prompt the Agent:

💬 List all of my datasets💬 Load the Berkeley Deep Drive dataset and show me a summary💬 Find near-duplicate images in my dataset💬 Select samples with confidence above 0.9💬 Compute CLIP embeddings and show me the embeddings panel colored by ground-truth detections.

Bring your own LLM: The Agent supports 100+ LLM providers, including Anthropic, OpenAI, and Google. Configure providers and API keys directly from the Agent settings panel.
Built-in skills: The Agent ships with a set of built-in skills covering the most common, including data import, enrichment, quality checks, model evaluation, and annotation. Think of a skill as a playbook that tells the Agent when to act and which steps to follow.
Custom skills: Teams can build their own skills with the new plugin skills framework. Plugins can now bundle agent-discoverable skills that are surfaced to the Agent and a new top-level fiftyone skills CLI command.
Guardrails by design: Operators can now declare a risk_level, giving the Agent’s guardrail system a way to classify high-impact actions before they run.

LIST THE SKILLS YOUR INSTALLED PLUGINS PROVIDE

# Discover every skill available to the Agentfiftyone skills list# Just the names, only from enabled pluginsfiftyone skills list --enabled --names-only

✏️ In-app annotation, reimagined: Versioned ontologies and AI-assisted masks

We’re making the hands-on annotation experience dramatically more powerful — and a lot faster. These annotation features are part of the open source FiftyOne 1.16.0 release and are included in Enterprise.

Annotation Ontologies: A new framework for declaring an append-only, versioned schema for your labels and their attributes. Ontologies live outside of any single dataset, so multiple datasets and fields can reference the same one by name. You can apply them to any field via the Schema Manager.
Conditional attributes: Ontologies support conditional attributes whose visibility is driven by composable When expressions (e.g., WhenEquals, WhenIn, and WhenAnd/WhenOr trees). Show annotators only the attributes that matter for the class they’re labeling. Hidden attributes are never written to the label.
Click-to-segment with SAM2: Annotators can now iteratively define masks using SAM2. Click positive and negative points and watch the mask boundary refine in real time.
Richer mask & polyline editing: Edit instance segmentation masks with pixel-level brush, polygon, and merge tools. Edit 2D polylines directly on a more stable annotation canvas.

📊 Observability: Full visibility into every delegated run

As teams run more delegated operations across more orchestrators, visibility becomes essential. FiftyOne Enterprise 2.19.0 gives platform teams a real window into what’s happening across their deployment.

Observability page: A new Observability page lets admin users monitor current and historical metrics and logs for deployed services, all in one place.
A full metrics dashboard on every run: The Run page for delegated operations now surfaces live system, database, and delegated-operation telemetry. It’s admin-only by default, with dataset-permission-based access controls available for non-admin users.
Live log streaming: Live log streaming during execution is now powered by a sidecar that captures stdout and stderr, surfacing in-progress logs, progress bars, and package versions directly in the UI.
Database panel: A new admin-only Database panel on the live runs metrics page gives deeper insight into database activity during execution.

🛠️ Quality of life & App experience

This release also includes a batch of targeted improvements that make everyday workflows smoother for both users and developers.

SAM3 in the Model Zoo: We’ve added Segment Anything 3, open-vocabulary instance segmentation that finds and segments all objects matching a text concept like “person” or “yellow school bus.”
Per-deployment model weights: Administrators can now configure where models in the Model Zoo source their weights from, including private buckets and air-gapped mirrors. This feature is ideal for secure, regulated, or offline deployments.
New built-in operators: A set of new built-in operators makes the App far more programmable and Agent-friendly. Get and set view filters, get and set panel state and data, list brain runs and model-evaluation scenarios, get field schemas, update the color scheme, and more.
More flexible panels: Register custom components in panel areas, including a resizable right sidebar and a wider default panel area, plus smoother synchronization between the Explore and Annotate viewports.

🛡️ Security and resilience

We’ve made several improvements to the platform’s security and reliability, including:

Sidecar CVEs: Resolved CVE-2026-24049, CVE-2026-23949, and CVE-2026-6357 in the delegated-operation sidecar.
Dependency upgrades: Updated pyjwt, ujson, and langchain-core to resolve outstanding security vulnerabilities.
CAS hardening: Fixed a CAS lockout in legacy mode caused by unrestricted onboarding flows, ensured labeler and service accounts now appear in the CAS audit table, and improved redirect preservation across client-side session and auth errors.
Infrastructure: Bumped the minimum supported Redis version.

Getting Started with 2.19.0

FiftyOne Enterprise 2.19.0 is available today! View the full release notes to see everything we’ve shipped.

New to FiftyOne Enterprise? FiftyOne Enterprise is the leading visual data engine for unlocking model performance. It unifies your data, your models, and your team in one secure platform. Get started today.

FiftyOne Enterprise 2.19.0 Released with the FiftyOne Agent and Next-Generation Annotation

Talk to a computer vision expert