Image annotation is the process of labeling images so a computer vision model can learn from them, by adding tags, boxes, masks, polygons, or keypoints that mark what each image contains and where. It is annotation applied specifically to image data.
Image annotation is the image-specific form of data annotation: taking raw images and attaching the labels a model needs to learn from them. Depending on the task, that label might be a single class tag for the whole image, a bounding box around each object, a pixel-precise mask, a polygon outline, or a set of keypoints.
The right type depends on what the model must do, classify, detect, or segment, and image annotation is the foundation of nearly every computer vision dataset, from ImageNet to autonomous-driving benchmarks.
Key takeaways
Image annotation is data annotation applied to images, and the label type depends on the task.
It spans classification tags, bounding boxes, masks, polygons, and keypoints.
It is the foundation of supervised computer vision, so the labels set the ceiling on model quality.
What image annotation provides
How image annotation relates to data annotation and video annotation.
How image annotation relates to data annotation and video annotation.
Term
Scope
Image annotation
Labeling image data specifically
Data annotation
The broader practice across all modalities
Video annotation
Labeling video, adds time and tracking
How it works
You define a schema, label the images against it (increasingly with model assistance like click-to-segment), review for quality, and export for training. The label types range from classification tags and bounding boxes to polygons, masks, and keypoints. FiftyOne wraps that loop: curate which images are worth labeling, route them to an annotation tool, and pull the labels back to inspect and compare against model predictions.
Why it matters
Image annotation is where most computer vision projects spend their time and budget, and where their accuracy ceiling is set. The most consequential image-annotation decision is choosing the label type, and teams often over-label. Annotating pixel masks when bounding boxes would train the model just as well burns budget for no gain, while under-labeling, boxes where the task truly needs shape, caps accuracy. Matching the annotation type to what the model actually needs, before scaling, is the cheapest large lever in the whole pipeline.
Frequently asked questions
What are the main types of image annotation?
Classification tags, bounding boxes, polygons, segmentation masks, and keypoints.
What is the difference between image annotation and data annotation?
Image annotation is data annotation applied specifically to images. Data annotation spans all modalities.
What tool do you use for image annotation?
A range of annotation tools, increasingly with model-assisted features like click-to-segment, plus FiftyOne to curate and review the labels.