Instance segmentation
Instance segmentation is an annotation and prediction task that outlines each individual object in an image at the pixel level, giving every object its own mask and identity. Unlike semantic segmentation, it tells two cars apart, combining the per-pixel detail of a mask with per-object separation.
In this article
Building visual or physical AI?
Let's talk.
What is instance segmentation?
Instance segmentation produces a separate pixel mask for every object, so the output is a set of individually identified objects, each with a class and its own boundary. It is essentially detection plus segmentation: the model both finds each object and traces its exact shape. Where semantic segmentation lumps all "person" pixels together, instance segmentation knows person 1 from person 2, which is what you need whenever counting or tracking individual objects matters.
Key takeaways
- Each object gets its own pixel mask and identity.
- It combines object separation, like detection, with exact shape, like a mask.
- It covers "things," countable objects, not amorphous "stuff" like sky or road, which is semantic territory.
How it works
Annotators outline or paint each object separately, often with click-to-segment assistance, then review for missed or merged objects. Evaluation matches predicted instances to ground truth by IoU, which surfaces objects that were merged, split, or missed.
Instance vs semantic vs panoptic segmentation
| Task | What it labels | Object identity? |
|---|---|---|
| Instance segmentation | Countable objects, one mask each | Yes, each object is separate |
| Semantic segmentation | Every pixel by class | No |
| Panoptic segmentation | Every pixel by class, plus instance IDs | Yes for objects, background gets a class only |
Where it's used
- Robotics and manipulation: separating individual items to grasp.
- Cell biology and medical imaging: counting and measuring individual structures.
- Retail and inventory: counting individual products.
Why it matters
Instance segmentation is the choice when you need both shape and count. Its hardest failure mode is not classification but instance separation. Touching or overlapping objects get merged into one mask or split into two, and standard mask IoU can look fine while the instance count is wrong. That is why crowded scenes, a pile of cells, a shelf of identical items, are where instance models break, and where targeted annotation of dense, overlapping examples pays off most.
Frequently asked questions
What is the difference between instance and semantic segmentation?
Instance separates individual objects, semantic labels every pixel by class without identity.
Is instance segmentation the same as object detection?
No, detection gives boxes, instance segmentation gives a precise mask per object.
What is panoptic segmentation?
A combination that labels every pixel by class and gives countable objects instance IDs.
Related terms
Learn more
See why pixel-precise masks beat boxes when shape and count matter in Preventing Your Vision AI Models From Failing.
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 3, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved