Point cloud annotation is the process of labeling 3D point cloud data, the set of points a sensor like LiDAR captures, by marking objects and regions within it, typically with 3D cuboids or per-point segmentation. It gives models spatial, depth-aware ground truth.
A point cloud is a collection of points in 3D space, each with a position and often intensity or color, produced by sensors like LiDAR or depth cameras. Point cloud annotation labels that data: drawing 3D cuboids around objects, assigning a class to each point through segmentation, or marking keypoints in space.
Unlike image annotation, the annotator works in three dimensions, rotating and slicing the scene to place labels accurately, and the points are sparse and unevenly distributed, dense up close, thin far away, which makes labeling distant or small objects genuinely hard.
Key takeaways
It labels 3D points from sensors like LiDAR, in three dimensions rather than on a flat image.
The main forms are 3D cuboids for objects and per-point segmentation for regions.
Sparsity and the 3D interface make it slower and harder than image annotation.
What point cloud annotation provides
Common forms of point cloud annotation.
Common forms of point cloud annotation.
Type
What it is
3D cuboids
Oriented boxes around objects, the most common form
Point cloud segmentation
A class label per point
Keypoints and polylines in 3D
Specific points or lines in space
Sensor fusion
Labeling with camera images aligned to the point cloud for context
How it works
Annotators view the cloud in 3D, often alongside fused camera frames, and place cuboids or paint point labels, rotating the scene to check fit from multiple angles. In FiftyOne, point clouds render natively in the 3D visualizer with their labels, so you can inspect cuboids and point segmentations from any viewpoint and compare them to model predictions.
Why it matters
Point cloud annotation is the ground truth behind autonomous driving and robotics, where a model must reason about real 3D space, not a 2D projection. The hardest part is not the geometry but the sparsity. A pedestrian 60 meters out may be a dozen points, indistinguishable from a pole or noise without the fused camera view. That is why labeling quality drops sharply with distance, why fusion with images is almost mandatory for far objects, and why distant-object recall is a known weak spot in 3D datasets that targeted annotation has to shore up.
Frequently asked questions
What is point cloud annotation used for?
Training 3D perception for autonomous vehicles, robotics, and mapping, where depth and space matter.
What types of labels go on a point cloud?
3D cuboids, per-point segmentation, and keypoints or polylines in space.
Why is point cloud annotation harder than image annotation?
It is done in 3D, and points are sparse and uneven, so distant or small objects are hard to identify without fused camera data.