Embedding pooling aggregates many embeddings into one, such as averaging token embeddings into a sentence embedding or frame embeddings into a video embedding. The pooling method determines what information the summary vector keeps and what it washes out.
Embedding pooling reduces a set of vectors to a single vector. Models and pipelines do it constantly: token embeddings pool into a sentence embedding, patch embeddings into an image embedding, frame embeddings into a clip embedding, segment embeddings into an episode summary. The pooled vector trades detail for compactness, one searchable point instead of many.
How you pool matters, because each method preserves different information.
Key takeaways
Pooling turns many embeddings into one summary vector.
Mean, max, and attention pooling preserve different information.
Pooled vectors trade fine-grained detail for compact searchability.
How pooling methods compare
Common pooling strategies and what they emphasize.
Common pooling strategies and what they emphasize.
Method
What it does
Mean pooling
Averages all vectors, capturing overall content
Max pooling
Keeps the strongest activation per dimension, emphasizing salient features
CLS / dedicated token
Uses a learned summary token trained to represent the whole input
Attention pooling
Learns weights so informative elements contribute more
How it works
Given a set of embeddings, pooling applies a permutation-invariant reduction, an average, a maximum, or a learned weighted sum, to produce one vector of the same dimensionality. Mean pooling is the robust default. Attention pooling learns to upweight the elements that matter, which helps when the signal is concentrated in a few tokens, frames, or segments.
Why it matters
Pooling decisions quietly shape retrieval quality at every granularity level: an episode embedding that mean-pools a thousand frames may bury the two seconds that made the episode interesting. Knowing what the pooling kept, and when to search segment-level instead, is key to building embedding indexes that find what you actually care about.
Frequently asked questions
When does mean pooling fail?
When the important content is a small fraction of the whole, an average dilutes it. Rare moments in long recordings are the classic case.
What is the alternative to pooling for long content?
Keep finer-grained embeddings, at the segment or frame level, and search those directly instead of one diluted summary.