Segmentation mask
A segmentation mask is an annotation that labels an object or region at the pixel level, marking exactly which pixels belong to it rather than enclosing it in a box. Masks can be semantic (a class per pixel) or instance (a separate mask per object).
In this article
Building visual or physical AI?
Let's talk.
What is a segmentation mask?
A segmentation mask assigns a label to individual pixels, so it captures an object's exact shape instead of just its bounding rectangle. It is usually stored as a binary or indexed raster the same size as the image, as a compact run-length encoding (RLE) like the one COCO uses, or as the polygon outline that rasterizes into it. Each mask carries a class, and depending on the task it may also carry an instance ID.
Key takeaways
- A mask captures exact shape at the pixel level, which a bounding box cannot.
- Masks are more expensive to label than boxes, often by roughly an order of magnitude.
- Semantic masks label every pixel by class, instance masks separate individual objects.
Types and formats
- Semantic, instance, and panoptic: class-per-pixel, per-object, or the two combined.
- Storage: binary or indexed raster, RLE (compact, used by COCO), or polygon outlines that rasterize to a mask.
- Drawn vs painted vs model-assisted: vertices, a brush, or increasingly a click-to-segment model.
How it works
Annotators paint or click to produce the mask, more and more with model assistance like SAM-style click-to-segment, then review the edges.
Before you train on instance masks, look at their edges
A segmentation mask marks exactly which pixels belong to an object. Instance masks go a step further and separate objects of the same class, so two overlapping people get two masks. A model learns the edges you give it, sloppy or not. The quiet problems:
- masks traced loosely around an object, so the edge includes background
- two touching objects merged into one mask
- an object with a box and a class label but no mask at all
In Voxel51, Open Images 10K has 10,000 images across 20 everyday classes with boxes and instance masks, and you can ask the Voxel51 Agent to find samples with potential annotation mistakes.
Open Images 10K
10,000 images
Open Images V7 images balanced across 20 everyday classes, with boxes, instance masks, image labels, points, and relationships.
Try this: Ask the Voxel51 Agent to find samples with potential annotation mistakes, then compare the masks and boxes in the images it returns.
Open the images in Voxel51
Free with a work or .edu email.
Go deeper
Mask vs box vs polygon
| Term | What it captures | Cost |
|---|---|---|
| Bounding box | A rectangle around the object | Low |
| Polygon | The object outline as vertices | Medium |
| Segmentation mask | Per-pixel membership | High |
Why it matters
Masks are what you need when shape matters, medical imaging, manufacturing defects, anything where a rectangle is too coarse. Mask quality lives or dies at the boundary. Most labeling error and most inter-annotator disagreement is on the edge pixels, and IoU on a thin or wispy object, hair, wires, transparent regions, punishes a few stray boundary pixels heavily. That is why boundary-aware review and model-assisted refinement matter so much for masks.
Frequently asked questions
What is the difference between a segmentation mask and a bounding box?
A box is a rectangle marking location, a mask labels the exact pixels of the object.
What is the difference between semantic and instance segmentation?
Semantic labels every pixel by class, instance gives each object its own mask.
How are masks stored?
As a raster, as run-length encoding (RLE), or as polygon outlines that rasterize to a mask.
Related terms
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 3, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved