3D annotation is the labeling of three-dimensional data, such as point clouds, depth scans, and 3D scenes, by marking objects and regions in space rather than on a flat image. It captures position, size, and orientation in three dimensions, most often with cuboids or point segmentation.
3D annotation is the umbrella term for labeling data that exists in three dimensions: LiDAR and depth-sensor point clouds, fused multi-sensor scenes, and reconstructed 3D models. Instead of drawing on a 2D image, the annotator works in space, placing oriented boxes (cuboids) around objects, assigning classes to individual points (point cloud segmentation), or marking keypoints and paths in 3D.
It is the foundation for any system that must understand where things are in the real world, which is why it dominates autonomous driving and robotics. Because it spans several data types and label forms, 3D annotation is best understood as the parent of more specific practices like point cloud annotation, LiDAR annotation, and cuboid labeling.
Key takeaways
3D annotation labels objects and regions in three-dimensional space, not on a flat image.
Its main forms are cuboids and point cloud segmentation, plus 3D keypoints and polylines.
It is the foundation of spatial perception for physical AI, and the umbrella over point cloud, LiDAR, and cuboid annotation.
What 3D annotation provides
The forms 3D annotation takes.
The forms 3D annotation takes.
Form
What it is
Cuboids and 3D bounding boxes
Oriented boxes around objects
Point cloud segmentation
A class per 3D point
3D keypoints and polylines
Points and lines located in space
Across data types
LiDAR, depth cameras, and fused multi-sensor scenes
How it works
Annotators work in a 3D viewer, rotating and slicing the scene to place labels accurately, often with fused camera views for context. In FiftyOne, 3D data and its labels render in the 3D visualizer, so you can inspect cuboids and point segmentations from any angle and compare them against model predictions.
Why it matters
3D annotation is what gives a model a sense of space, the difference between recognizing a car in a photo and knowing where it is and which way it is moving in the world. 3D labels are far more expensive than 2D, often several times the cost, because the annotator must get position, size, and orientation right in a sparse, rotatable scene. So the curation question, which frames and objects are actually worth labeling in 3D, matters even more here than in images. Labeling 3D data indiscriminately is one of the fastest ways to burn an annotation budget.
Frequently asked questions
What is 3D annotation?
Labeling three-dimensional data like point clouds and 3D scenes by marking objects and regions in space.
What are the main types of 3D annotation?
Cuboids, point cloud segmentation, and 3D keypoints or polylines.
How is 3D annotation different from 2D?
It works in space with position and orientation, on sparse rotatable data, which makes it more costly and complex than labeling flat images.