What Is Keypoint Annotation?

What is keypoint annotation?

Keypoint annotation places individual labeled points at meaningful locations on an object: the elbows and knees of a person, the corners of a license plate, the tip of a tool. Each keypoint has an (x, y) coordinate, a label saying which point it is, and usually a visibility flag noting whether it is visible, occluded, or absent. Keypoints are often grouped into an ordered skeleton, a fixed template like the 17-point COCO human pose layout, so the connections between points carry meaning too.

Key takeaways

Keypoints mark specific labeled locations, not an object's full extent or shape.
Each point carries a coordinate, an identity, and a visibility state.
They are usually arranged in a fixed skeleton, which is what makes pose estimation possible.

Types and examples

Human pose: body joints, the basis of pose estimation.
Facial landmarks: eyes, nose, and mouth corners for face analysis.
Object keypoints: fixed reference points on rigid objects for orientation or measurement.
2D vs 3D: image-plane coordinates, or points located in 3D space.

Keypoints vs boxes vs masks

Comparison of annotation types and what they capture

Comparison of annotation types and what they capture
Annotation type	What it captures
Bounding box	Where an object is, a rectangle around it
Segmentation mask	An object's exact shape, pixel by pixel
Keypoint annotation	Specific points and the structure between them, the pose

Where it's used

Sports and health: movement and gait analysis.
AR and avatars: driving rigs from body or face points.
Robotics: locating grasp points or object orientation.

Why it matters

Keypoints capture structure and motion that a box or mask misses, the difference between "a person is here" and "this is what they are doing." The visibility flag is the part teams underuse and mislabel. An occluded joint that gets dropped or guessed teaches the model the wrong thing, and inconsistent visibility conventions across annotators are a leading source of noise in pose datasets. Defining occluded-versus-absent rules up front matters more than the exact point placement.

Frequently asked questions

What is the difference between keypoint annotation and pose estimation?

Keypoint annotation is the labeling, pose estimation is the model task that predicts those keypoints.

What is a skeleton in keypoint annotation?

A fixed template of named points and the connections between them, like the 17-point COCO human layout.

What does the visibility flag mean?

Whether a keypoint is visible, occluded, or absent, which affects how the model should treat it.

Learn more

Get the fundamentals in the 2026 guide to data annotation.

Jesse Mostipak is the SEO and Content Manager at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn

Keypoint annotation

Building visual or physical AI? 
Let's talk.