Image classification is a computer vision task, and annotation type, that assigns one or more class labels to an entire image, identifying what it contains overall without locating where. It is the simplest and cheapest form of labeling, a tag per image rather than a box or mask.
Image classification answers "what is this image" by assigning it a category, cat, x-ray, or defect, from a fixed set of classes. As an annotation type it is the lightest one: the annotator or model tags the whole image rather than marking any region, so there is no localization, just a label.
Classification can be single-label, exactly one class per image, or multi-label, several tags at once like "outdoor" and "night." Because the labels are so cheap to produce, classification datasets scale to enormous sizes, which is why it has long been the entry point for both annotation and model training.
Key takeaways
Image classification labels the whole image with a class, with no location information.
It is the cheapest annotation type, a tag rather than a box or mask.
It can be single-label or multi-label, depending on whether an image can belong to more than one class.
What image classification provides
How classification compares to detection and segmentation.
How classification compares to detection and segmentation.
Task
What it outputs
Localization
Image classification
A class for the whole image
None
Object detection
Boxes around objects
Coarse, boxes
Segmentation
Per-pixel classes
Exact
How it works
An annotator or model assigns the class, and that label trains a classifier to predict the category for new images. Classes can be binary, multi-class, or organized hierarchically in a taxonomy (vehicle, then car, then sedan). In FiftyOne, classifications render as sample-level tags, and evaluation compares predicted classes to ground truth, with a confusion matrix showing which classes the model swaps.
Why it matters
Classification is the foundation task and the cheapest place to start, but its simplicity hides a trap. Because the label is image-wide, classifiers latch onto whatever correlates with the class, including the background, the camera, or a watermark, rather than the object itself. The classic failure is a model that "recognizes" a class by its setting, cows on grass or ships on water, and collapses when the context changes, which is why curating diverse, decorrelated examples matters more for classification than raw volume.
Frequently asked questions
What is the difference between image classification and object detection?
Classification labels the whole image with a class. Detection also locates objects with boxes.
What is multi-label classification?
Assigning more than one class to a single image, like "beach" and "sunset."
Why do classifiers learn the background?
Because the label covers the whole image, so any feature that correlates with the class, including context, can be used.