SAM 2 (Segment Anything Model 2)
SAM 2 is Meta AI's 2024 successor to the Segment Anything Model that extends promptable segmentation from images to video, tracking an object across frames from a single click using a memory mechanism. It segments in both images and video in real time.
In this article
Building visual or physical AI?
Let's talk.
What is SAM 2?
SAM 2 generalizes SAM from still images to video. You prompt an object once, a click or box on one frame, and SAM 2 propagates its mask through the rest of the video, following the object as it moves, deforms, and is briefly occluded. It does this with a memory module that remembers the object across frames, and it still handles single images as a special case.
Like SAM, it is promptable and zero-shot, so it segments objects it was not specifically trained on, now across time. It was trained on the large SA-V video segmentation dataset and runs fast enough for interactive use.
Key takeaways
- SAM 2 extends SAM to video, propagating a mask across frames from one prompt.
- A memory mechanism lets it track an object through motion and brief occlusion.
- It handles both images and video in one model, in real time.
What SAM 2 provides
| Capability | SAM | SAM 2 |
|---|---|---|
| Media | Images | Images and video |
| Tracking | None | Across frames, via memory |
| Prompt | Click, box, region | Same, plus propagation |
How it works
You annotate an object on one frame and SAM 2 carries the mask forward, with the chance to correct it where it drifts.
Why it matters
SAM 2 attacks the most painful annotation cost in video: masking the same object frame after frame. The win is interactive, not just about accuracy. Because SAM 2 propagates from a single prompt, the annotator's job becomes spotting and fixing where the mask drifts, fast motion, occlusion, a similar object nearby, rather than drawing every frame. The leverage on video labeling is enormous, but the human review still concentrates exactly where tracking is hard, which is why correction tooling matters as much as the model.
Frequently asked questions
What is the difference between SAM and SAM 2?
SAM segments still images. SAM 2 adds video, propagating a mask across frames with memory.
Can SAM 2 track objects?
Yes, its memory mechanism follows an object across frames from a single prompt.
Does SAM 2 still work on images?
Yes, it handles both images and video in one model.
Related terms
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 3, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved