The Segment Anything Model (SAM) is a foundation model for image segmentation, released by Meta AI in 2023, that produces a mask for any object from a simple prompt like a click or a box, without being trained on that object's class. It is widely used to accelerate annotation.
SAM is a promptable segmentation model: you give it an input, a point, a box, or a rough region, and it returns a precise mask for the object there, even for categories it never saw labeled. It was trained on SA-1B, a dataset of over a billion masks across 11 million images, which gives it broad, zero-shot generalization to new images and objects.
Rather than classifying what an object is, SAM focuses on cutting it out cleanly, which is why it became a backbone for annotation tooling: it turns a single click into a high-quality mask.
Key takeaways
SAM is a foundation model that segments any object from a prompt, a click, box, or point, zero-shot.
It was trained by Meta AI on the billion-mask SA-1B dataset for broad generalization.
It powers click-to-segment and model-assisted labeling, dramatically speeding up mask annotation.
What SAM provides
How SAM compares to a trained segmentation model and to its video successor, SAM 2.
How SAM compares to a trained segmentation model and to its video successor, SAM 2.
Model
What it segments
Prompt
SAM
Any object in an image, zero-shot
Point, box, or region
A trained segmentation model
Only its trained classes
None, automatic
SAM 2
Objects across video and images
Point or box, with memory across frames
How it works
SAM separates a one-time image encoding from a fast prompt step, so once an image is encoded, each click returns a mask almost instantly. That design is what makes it practical for interactive annotation (click an object, get a mask), for auto-labeling and pre-labeling at scale, and as a building block for downstream pipelines and its successors, SAM 2 and SAM 3. In FiftyOne, SAM is available through the model zoo and integrations, so you can run it to generate masks, store them as labels, and review or correct them in the app.
Why it matters
SAM reset the economics of mask annotation, the most expensive label type, by turning careful pixel painting into a click plus a correction. The catch is that SAM segments but does not name. It tells you where an object is, not what it is, so it does not replace classification or your label schema, it accelerates the boundary work while a human or another model still supplies the class. Teams that expect SAM to "label the dataset" miss that it solves the geometry, not the taxonomy.
Frequently asked questions
What does SAM do?
It produces a segmentation mask for any object from a prompt like a click or box, without class-specific training.
Does SAM classify objects?
No, it segments. It outputs the mask, not the class, so a human or another model assigns the label.
What is SAM used for in annotation?
Click-to-segment and auto-labeling, turning a click into a high-quality mask to speed up labeling.