Point cloud

A point cloud is a collection of individual points in 3D space that, taken together, trace out the shape of a scene or object. Each point carries at least its position, given as x, y, and z coordinates, and often additional attributes such as the intensity of a returned laser pulse or a color sampled from a camera. There is no built-in surface or connectivity between points, just a large set of measured locations, which is why a point cloud looks like a fine spray of dots that resolves into recognizable structure when you have enough of them.

Point clouds are the natural output of sensors that measure distance directly, most notably lidar, which sweeps a laser across a scene and records where each pulse bounces back, and depth cameras, which estimate distance for each pixel. The result is a spatially accurate but sparse and unordered representation of the world. This combination of precision and sparseness is what makes point clouds both powerful and challenging to work with, and it shapes how objects are detected, tracked, and labeled within them.

A point cloud is generated by measuring the distance and direction to many points on the surfaces around a sensor, then converting those measurements into 3D coordinates in a common frame. Lidar does this by timing laser pulses, while depth cameras infer distance from patterns of light or from multiple views. Because the points are unordered and unevenly distributed, being dense where surfaces are close and sparse where they are far, working with them relies on spatial methods that reason about neighborhoods of points rather than a fixed grid. When point clouds from multiple sensors or multiple moments are combined, accurate calibration and pose estimation are what let them register into a single consistent cloud.

Point clouds give physical AI systems a direct, geometrically faithful sense of 3D structure, which is difficult to obtain from flat images alone. They matter to anyone working on perception for autonomous vehicles and robots, because so much of understanding a scene, such as how far away an obstacle is and how large it is, depends on accurate 3D measurement. The way objects are annotated in point clouds, most commonly with 3D bounding boxes, directly determines what a model can learn to perceive, which ties point cloud quality closely to downstream safety and reliability.

An image is a dense 2D grid of pixels capturing color and texture, while a point cloud is a set of 3D points capturing geometry. Point clouds directly encode distance and shape, which images do not, but they lack the dense appearance detail that images provide.

Lidar is the most common source, producing points by timing reflected laser pulses. Depth cameras also produce point clouds by estimating distance per pixel, and point clouds can be built from multiple camera views as well.

Points are measured only where surfaces reflect the sensor's signal, and they are denser for nearby surfaces and sparser for distant ones, so coverage is uneven. There is also no inherent ordering or connectivity between points, which is why processing them relies on spatial rather than grid-based methods.

Multimodal sensor data, Sensor calibration, 3D bounding box / cuboid, Sensor fusion, Physical AI.