Object detection
Object detection is a computer vision task that locates and classifies objects in an image or video, drawing a bounding box around each one and assigning it a class. It answers both what is present and where, unlike classification, which only says what.
In this article
Building visual or physical AI?
Let's talk.
What is object detection?
Object detection combines two questions into one task: what objects are in the image, and where is each one. A detector outputs a set of bounding boxes, each with a class label and a confidence score, so a single image yields many detections.
It sits between image classification, which labels the whole image with no location, and segmentation, which labels exact pixels: detection gives coarse location, a box, for every object, which is enough for a huge range of applications, counting, tracking, and triggering actions. Its training data is bounding-box annotation, the boxes a model learns to predict.
Key takeaways
- Object detection finds and classifies each object, outputting a box, a class, and a confidence.
- It is the middle ground, more than classification, which has no location, less than segmentation, which has exact shape.
- It learns from bounding-box annotation and is scored with IoU and mean average precision (mAP).
What object detection provides
| Task | Output | Location |
|---|---|---|
| Image classification | A class for the image | None |
| Object detection | Boxes with classes | Coarse, boxes |
| Segmentation | Per-pixel classes | Exact |
How it works
A detector proposes boxes and classifies them, keeping those above a confidence threshold.
Why it matters
Object detection is one of the most widely deployed CV tasks, the workhorse behind surveillance, retail analytics, driving, and inspection. Detection metrics hide where models fail. mAP is an average over classes and thresholds, so a model can post a strong number while quietly missing small objects, rare classes, or crowded scenes. The useful move is to slice evaluation, by object size, class, and scene density, rather than trusting the headline mAP, because that is where you find the failures that matter and the data you need to annotate next.
Frequently asked questions
What is the difference between object detection and image classification?
Classification labels the whole image. Detection also locates each object with a box.
What is the difference between detection and segmentation?
Detection gives a box per object. Segmentation gives exact per-pixel shape.
How is object detection evaluated?
With IoU to match boxes and mean average precision (mAP) across classes and thresholds.
Related terms
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 3, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved