Optical flow

Optical flow describes how each pixel appears to move from one video frame to the next. The result is a dense field of motion vectors, one per pixel, showing the direction and speed of apparent movement across the image. That movement can come from objects moving, the camera moving, or both.

It is a fundamental cue for understanding motion in video, distinct from tracking specific objects.

Classic methods estimate flow by assuming pixel brightness is roughly constant between frames and solving for the motion that explains the change, while modern methods learn to predict flow directly with neural networks and handle large, complex motions better. The output flow field can then drive downstream tasks such as tracking objects, interpolating frames, or stabilizing shaky footage.

Motion is one of the strongest signals in video, and optical flow makes it explicit and usable. It supports object tracking, video annotation workflows like interpolation, and any analysis that depends on how a scene changes over time, which is common in visual AI.

Optical flow estimates motion for every pixel, while object tracking follows specific identified objects across frames, often using flow as one of its cues.

Any apparent motion between frames, whether from moving objects, a moving camera, or both.

Object tracking, Multi-object tracking (MOT), Video annotation, Depth estimation.