Click-to-segment is an interactive annotation technique where a single click on an object produces a full segmentation mask, with a model inferring the object's boundary from that one prompt. It replaces manual pixel painting or polygon drawing with a click and a quick correction.
Click-to-segment is the interaction at the heart of modern model-assisted labeling. Instead of tracing an outline vertex by vertex or painting pixels, the annotator clicks on an object and a segmentation model, typically a SAM-style foundation model, returns a complete mask.
Extra clicks refine it: a positive click adds a missed region, a negative click removes spillover. The result is mask-quality segmentation at a fraction of the effort, which is why it has become the default way to create masks in most annotation tools.
Key takeaways
One click yields a full mask, with extra clicks to refine.
It is the interaction layer that turns segmentation models like SAM into a labeling tool.
It collapses the cost of mask annotation, the most expensive label type, to clicks plus corrections.
What click-to-segment provides
The pieces behind a click-to-segment workflow.
The pieces behind a click-to-segment workflow.
Element
What it does
Positive and negative clicks
Add or remove regions to shape the mask
Box or scribble prompts
Some tools accept a rough box or stroke instead of a point
Model behind it
A promptable segmentation model, SAM, SAM 2, or SAM 3, does the inference
Propagation
In video, the mask carries across frames, the SAM 2 and SAM 3 extension
How it works
A click becomes a prompt to a segmentation model, whose mask is written in as a label the annotator can accept or refine. In FiftyOne, SAM-family models are integrated, so click-to-segment style workflows can generate masks that you store, review, and correct.
Why it matters
Click-to-segment is the single biggest reason mask annotation stopped being prohibitively slow, because it changed segmentation from a per-pixel chore into a per-object click. The catch is the same automation bias that haunts all assisted labeling. A one-click mask looks finished, so annotators stop scrutinizing the boundary, and subtle errors, a sliver of background or a merged neighbor, slip through because the result feels authoritative. The speed is real, but quality still depends on reviewing the edges the model got almost, but not quite, right.
Frequently asked questions
What is click-to-segment?
An annotation technique where clicking an object produces a segmentation mask from a model, refined with more clicks.
What model powers click-to-segment?
Usually a SAM-family promptable segmentation model.
How is it different from polygon annotation?
Polygon annotation places vertices by hand. Click-to-segment generates the mask from a click and you correct it.