Video annotation interpolation is a labeling technique that fills in an object's annotations on the frames between two manually labeled keyframes by estimating its position in each intermediate frame. It lets annotators label sparse keyframes and have the rest generated automatically.
Interpolation is the workhorse that makes video annotation affordable. Instead of drawing a box on all 1,800 frames of a clip, an annotator labels the object on a handful of keyframes, the start, the end, and a few turning points, and the tool computes its position on every frame in between.
The simplest form is linear interpolation, where the box moves in a straight line at a steady rate between keyframes, and the annotator adds keyframes wherever the motion is not linear. The result is full per-frame labels from a fraction of the manual work.
Key takeaways
Interpolation generates labels on intermediate frames from manually labeled keyframes.
It turns labeling thousands of frames into labeling a few, with the tool filling the gaps.
Accuracy depends on keyframe placement, so you add keyframes wherever motion changes.
What interpolation provides
How interpolation fills the frames between keyframes.
How interpolation fills the frames between keyframes.
Step
What happens
Keyframes
The annotator labels an object on selected frames
Interpolation
The tool estimates the object's box or mask on the frames between them, usually linearly
Correction
The annotator adds keyframes where the interpolated path drifts from the object
Interpolation vs propagation
Interpolation is geometric estimation between human labels, propagation (SAM 2) predicts frames with a model
How it works
You set keyframes and the labels fill in between, then scrub the timeline and fix where they drift. In FiftyOne, video labels are stored per frame, so interpolated and corrected annotations live alongside each other and can be reviewed across the whole clip.
Why it matters
Interpolation is the difference between video annotation being feasible and being prohibitively expensive. Its quality is entirely a function of where keyframes go, and the common mistake is spacing them evenly. Motion is not uniform, an object sits still, then darts, so evenly spaced keyframes over-label the still parts and under-label the fast ones, producing boxes that lag or overshoot exactly during the moments that matter. Smart keyframing follows the motion, dense where it changes, sparse where it does not.
Frequently asked questions
What is interpolation in video annotation?
Generating an object's labels on frames between keyframes by estimating its position, so you only hand-label keyframes.
What is the difference between interpolation and propagation?
Interpolation estimates geometrically between human-labeled keyframes. Propagation uses a model to predict frames, like SAM 2.
How many keyframes do you need?
As many as the motion requires, more where the object's path changes, fewer where it moves steadily.