Depth estimation

Depth estimation predicts the distance from the camera to every point in view, producing a depth map where each pixel carries a distance value. It recovers the 3D structure that an ordinary image flattens away, which is what lets a system reason about how far apart things are and where surfaces sit in space.

It can be done from a single image, from a stereo pair, or by combining cameras with active sensors.

Stereo methods triangulate depth from the disparity between two calibrated cameras, much as two eyes do. Monocular methods learn to infer depth from a single image using cues the model has learned, such as size, texture, and perspective. Depth can also be measured directly by sensors like LiDAR and then fused with images, and the resulting depth maps often feed 3D reconstructions and point clouds.

Any system that must move through or manipulate the world needs to know distances, which makes depth estimation foundational for robotics, autonomous vehicles, and augmented reality. It bridges 2D perception and the 3D representations, like point clouds, that physical AI depends on.

Stereo uses two cameras and triangulation, while monocular infers depth from a single image using learned visual cues, which is harder but needs only one camera.

A depth map stores distance per pixel in image space, while a point cloud is a set of 3D points, which a depth map can be converted into using camera geometry.

Point cloud, SLAM (simultaneous localization and mapping), Neural radiance field (NeRF), Sensor fusion.