Pose estimation is a computer vision task that predicts the configuration of a body or object by locating its keypoints, such as joints, and inferring how they connect. It turns an image or video into a skeleton, capturing posture and movement rather than just an object's location.
Pose estimation locates the meaningful points of a subject, the joints of a person, the parts of a hand, the reference points on an object, and arranges them into a structured skeleton. The output is not a box or a class but a set of connected keypoints that describe configuration: where the elbows, knees, and shoulders are, and therefore what the body is doing.
It can be 2D, points on the image, or 3D, points in space, and it relies on keypoint annotation for its training data, the labeled skeletons a model learns to predict.
Key takeaways
Pose estimation predicts keypoints and their connections, the skeleton, not just object location.
It captures posture and motion, which boxes and masks cannot.
It is the model task that learns from keypoint annotation, the predict-and-label sides of the same thing.
What pose estimation provides
Common variations of pose estimation.
Common variations of pose estimation.
Variation
Options
2D vs 3D pose
Keypoints on the image plane, or located in space
Single vs multi-person
One subject, or many in a scene
Subject
Human, hand, face, or object, each with its own skeleton
Top-down vs bottom-up
Detect each subject then find its keypoints, or find all keypoints then group them
How it works
A model locates the keypoints of a subject and connects them into a skeleton, either top-down (detect each subject, then find its keypoints) or bottom-up (find all keypoints, then group them by subject). The training data is keypoint annotation, where each labeled point carries a visibility flag marking whether it is visible, occluded, or absent.
Why it matters
Pose estimation is how machines read body language and movement, the layer of understanding above "an object is here," and it powers sports and health (form analysis, rehabilitation, gait), AR and gaming (driving rigs from body or hand pose), and robotics (estimating object orientation for grasping). Its accuracy is dominated by occlusion and self-occlusion, because a turned torso hides an arm and the model must infer the hidden joint. This is exactly why the visibility flag in keypoint annotation matters so much: models trained without consistent occluded-versus-absent labels learn to hallucinate joints, and most real-world pose errors trace back to how occlusion was handled in the labels, not to the model architecture.
Frequently asked questions
What is the difference between pose estimation and keypoint annotation?
Pose estimation is the model task that predicts keypoints. Keypoint annotation is the labeling that trains it.
What is the difference between 2D and 3D pose estimation?
2D locates keypoints on the image. 3D locates them in space, which handles occlusion and viewpoint better.
What is the output of pose estimation?
A set of connected keypoints, a skeleton, describing the subject's configuration.