What if you could just ask your computer-vision dataset a question — in plain English — and watch it filter itself?
We built exactly that: a FiftyOne panel where you type a question and the App shows you what you asked for. No query code, no docs-diving. Under the hood, an open-weight coding model writes the FiftyOne query, runs it, fixes its own mistakes, and hands back a filtered view. Here's what makes it tick.
Key takeaways
The Talk to Your Dataset panel is a FiftyOne plugin that converts a plain-English question into a FiftyOne view query, runs it, and returns a filtered view of the dataset in the App.
Laguna S 2.1 is Poolside's open-weight coding model, 118B total parameters with roughly 8B active per token, released under the OpenMDW-1.1 license.
Laguna has no vision capability. It reasons over the dataset schema and Python tracebacks, not pixels, and rewrites failed queries until they run.
The agent routes across three tools: schema-based view queries, FiftyOne Brain for uniqueness and near-duplicates, and a vision-language model for questions that require actually looking at the image.
Because the agent re-reads the schema at query time, it adapts to a new dataset with zero prompt changes.
What is Laguna S 2.1?
Laguna S 2.1 is Poolside's open-weight foundation model built for agentic coding — writing code, running it, reading the result, and persisting until the job is done.
Small but mighty. 118B parameters total, but a Mixture-of-Experts design activates only ~8B per token. On agentic benchmarks like Terminal-Bench 2.1 and SWE-Bench Pro it matches or beats models several times its size.
Open weights. Released on Hugging Face under the Linux Foundation's permissive OpenMDW-1.1 license — use it, modify it, self-host it, even commercially.
Runs on a desktop. Compact enough to run on a single NVIDIA DGX Spark, or reach it through an OpenAI-compatible endpoint on OpenRouter (free and paid tiers, up to a 1M-token context).
Built to not give up. Poolside specifically trained it to check its work and revise failed approaches — the exact behavior our demo leans on.
Here’s something interesting. Laguna can't see. It's a text-only model with zero vision capability. That turns out to be the whole point.
What is FiftyOne?
FiftyOne is the open-source toolkit from Voxel51 for annotating and curating vision data and evaluating models. It gives you a fast, visual App to explore images and videos, slice and filter by labels and metadata, evaluate model predictions, find failure modes, and curate data — all backed by a powerful Python query API and an extensible plugin system.
That query API is expressive, but it's still code. If you don't already know the idioms, "find every image with more than five people where at least one box is tiny" is a puzzle. That's the gap we wanted to close.
How it works: Laguna reasons about code. FiftyOne handles the pixels
Laguna never looks at a single image. It reasons over the dataset schema and the Python errors it gets back — exactly like an engineer who's never seen your photos but knows the API cold. You ask a question; it writes a FiftyOne view query; if the query throws, it reads the traceback and rewrites it until it works. That self-correction loop is the demo's beating heart, and it's the behavior Laguna was trained for.
It's more than a single trick. We grew the agent into a little router across three tools:
View queries: filtering on the schema
Anything expressible over the schema (object counts, classes, confidence, bbox size).
"Show me images with more than 5 people where at least one person is small — under 5% of the image area."
"Show me the 20 most unique images in the dataset."
A vision model: delegating perception to a VLM
For questions that genuinely need eyes ("find images taken outdoors"), Laguna delegates perception to a VLM, then filters on the result. It orchestrates; the VLM sees.
"Find images taken outdoors."
And it lives inside the App. The whole thing is packaged as a FiftyOne plugin panel, so end users never touch Python — they just type and hit Run query.
Laguna S 2.1 is a small, cheap, open-weight coding model that makes a surprisingly strong dataset orchestrator. It reasons over schema and errors, picks the right tool, and writes the glue — and pairs cleanly with a vision model for the parts that need perception.
How the query agent routes three kinds of questions, and what runs behind each one.
How the query agent routes three kinds of questions, and what runs behind each one.
You ask
Tool the agent picks
What runs under the hood
"Show me images with more than 5 people where at least one person is small, under 5% of the image area."
View query
A FiftyOne view built from the dataset schema, filtering on object counts and bounding box area
"Show me the 20 most unique images in the dataset."
FiftyOne Brain
An embedding-based uniqueness score, sorted and limited to 20 samples
"Find images taken outdoors."
Vision-language model
Laguna delegates perception to a VLM, then filters the dataset on the VLM's output
Where to take it next? Wire FiftyOne Brain's similarity and uniqueness into the router for richer "find me..." requests, and pair Laguna with a vision-language model as a first-class tool so pixel-level questions and code-level orchestration live side by side.
Type a question. Watch your dataset answer. That's the future of data curation — and it's open source, top to bottom.
Build your own agent: clone the demo notebook, point it at coco-2017, bdd100k, or your own dataset — the agent re-reads the schema and adapts with zero prompt changes.
Frequently asked questions
Laguna S 2.1 is Poolside's open-weight foundation model built for agentic coding. It has 118B total parameters with a Mixture-of-Experts design that activates roughly 8B per token, and it is released on Hugging Face under the Linux Foundation's permissive OpenMDW-1.1 license, which allows commercial use, modification, and self-hosting.
No. Laguna is text-only with zero vision capability. It reasons over the dataset schema and the Python errors it gets back, then delegates any question that genuinely requires perception to a vision-language model.
When a generated query throws an exception, the agent reads the traceback and rewrites the query, repeating until it runs. Poolside trained Laguna specifically to check its work and revise failed approaches, which is the behavior the demo relies on.
No. The agent is packaged as a FiftyOne plugin panel, so you type a question in the App and click Run query.
It is compact enough to run on a single NVIDIA DGX Spark, or you can reach it through an OpenAI-compatible endpoint on OpenRouter, which offers free and paid tiers with up to a 1M-token context.
Yes. The agent re-reads the schema at query time, so pointing the notebook at coco-2017, bdd100k, or your own dataset requires no prompt changes.
