What is a patch embedding?
A patch embedding turns a small square of pixels, commonly 16 by 16, into a vector. A Vision Transformer begins by cutting an image into a grid of such patches and linearly projecting each into an embedding, producing a sequence of patch tokens the transformer can process exactly as it processes words. Positional encodings are added so the model knows where each patch came from.
Patch embeddings are internal representations, distinct from the single image-level embedding a model outputs for search and similarity.
Key takeaways
- Patch embeddings represent small fixed-size pieces of an image.
- They are the tokens a Vision Transformer's attention operates on.
- They are internal, unlike the image-level embeddings used for search.
How it works
Each patch is flattened into a vector of raw pixel values and multiplied by a learned projection to produce its embedding. The full sequence, plus positional information, flows through the transformer's attention layers, where patches exchange context. Pooling the final patch representations, or reading a dedicated classification token, yields the whole-image embedding.
Why it matters
Patch embeddings are the bridge that let transformers, built for token sequences, consume images at all. Their granularity also enables dense tasks: because every patch carries its own representation, models can localize, segment, and ground language to specific image regions rather than the image as a whole.
Frequently asked questions
How is a patch embedding different from an image embedding?
A patch embedding represents one small region and lives inside the model, while an image embedding summarizes the whole image for downstream use.
Why do Vision Transformers use patches?
Processing every pixel as a token would be prohibitively expensive. Patches shrink the sequence to a manageable length while preserving spatial detail.
Related terms
Go deeper