Keypoint annotation is an annotation type that marks specific points of interest on an object, such as joints, facial landmarks, or corners, each placed at an exact coordinate and often connected into a skeleton. It captures pose and structure rather than an object's full outline.
Keypoint annotation places individual labeled points at meaningful locations on an object: the elbows and knees of a person, the corners of a license plate, the tip of a tool. Each keypoint has an (x, y) coordinate, a label saying which point it is, and usually a visibility flag noting whether it is visible, occluded, or absent. Keypoints are often grouped into an ordered skeleton, a fixed template like the 17-point COCO human pose layout, so the connections between points carry meaning too.
Key takeaways
Keypoints mark specific labeled locations, not an object's full extent or shape.
Each point carries a coordinate, an identity, and a visibility state.
They are usually arranged in a fixed skeleton, which is what makes pose estimation possible.
Types and examples
Human pose: body joints, the basis of pose estimation.
Facial landmarks: eyes, nose, and mouth corners for face analysis.
Object keypoints: fixed reference points on rigid objects for orientation or measurement.
2D vs 3D: image-plane coordinates, or points located in 3D space.
Keypoints vs boxes vs masks
Comparison of annotation types and what they capture
Comparison of annotation types and what they capture
Annotation type
What it captures
Bounding box
Where an object is, a rectangle around it
Segmentation mask
An object's exact shape, pixel by pixel
Keypoint annotation
Specific points and the structure between them, the pose
Where it's used
Sports and health: movement and gait analysis.
AR and avatars: driving rigs from body or face points.
Robotics: locating grasp points or object orientation.
Why it matters
Keypoints capture structure and motion that a box or mask misses, the difference between "a person is here" and "this is what they are doing." The visibility flag is the part teams underuse and mislabel. An occluded joint that gets dropped or guessed teaches the model the wrong thing, and inconsistent visibility conventions across annotators are a leading source of noise in pose datasets. Defining occluded-versus-absent rules up front matters more than the exact point placement.
Frequently asked questions
What is the difference between keypoint annotation and pose estimation?
Keypoint annotation is the labeling, pose estimation is the model task that predicts those keypoints.
What is a skeleton in keypoint annotation?
A fixed template of named points and the connections between them, like the 17-point COCO human layout.
What does the visibility flag mean?
Whether a keypoint is visible, occluded, or absent, which affects how the model should treat it.