A bounding box is an annotation type that marks an object's location in an image or video with a rectangle, usually stored as corner coordinates or as an origin point with width and height. It is the most common label for object detection.
A bounding box is the simplest way to localize an object: a rectangle drawn tightly around it. It is typically axis-aligned and stored either as two corners (x_min, y_min, x_max, y_max) or as an origin plus size (x, y, width, height). Each box carries a class label, and a single image can hold many boxes. Boxes are the standard target for object detection, where the model learns to regress the rectangle and classify what is inside it.
Key takeaways
The bounding box is the cheapest and most common annotation type, which makes it the default for object detection.
It captures location and extent but not shape or orientation, so a tight box around an angled object still contains background.
Box accuracy is scored with IoU, the overlap between the predicted and ground-truth rectangle.
Types and variations
Axis-aligned vs rotated (oriented): a rotated box adds an angle to fit tilted objects.
2D vs 3D: the 3D analog is the cuboid, which adds depth and orientation.
Tight vs amodal: whether the box hugs only the visible pixels or the object's full estimated extent.
How it works, and how FiftyOne fits
Annotators draw the rectangle, often with snapping or model-assisted suggestions, and assign a class. In FiftyOne, detections render as boxes over the image with their labels and confidence, so you can filter, sort, and compare ground truth against model predictions to surface misses and false positives.
Bounding box vs related labels
How a bounding box compares to the other ways of localizing an object.
How a bounding box compares to the other ways of localizing an object.
Term
What it captures
When to use
Bounding box
2D rectangle: location and extent
Object detection in images and video
Polygon or mask
Exact object shape at the pixel level
When shape matters
Cuboid
3D box: location, size, and orientation
LiDAR and 3D scenes
Rotated (oriented) bounding box
A 2D box with an angle, to fit tilted objects
Aerial imagery, text, densely packed or angled objects
Keypoints
Specific points and the structure between them, not extent
Pose and landmark tasks
Why it matters
Boxes are where most detection datasets and models start, because they are fast to draw and cheap to scale. A tight axis-aligned box around a diagonal or thin object, a cyclist's arm, a power line, can be mostly background, so the model trains on noisy positives. That is exactly when teams move to rotated boxes or segmentation. IoU thresholds also make box evaluation unforgiving, a model can locate an object yet still fail the 0.5 IoU bar on a loose box.
Frequently asked questions
How is a bounding box stored?
Either as corner coordinates (x_min, y_min, x_max, y_max) or as an origin point plus width and height.
What is the difference between a bounding box and a mask?
A box marks location with a rectangle, a mask marks exact shape at the pixel level.
How is bounding box accuracy measured?
With IoU, the overlap between the predicted and ground-truth box.