Contrastive learning

Contrastive learning trains a model to organize an embedding space by similarity. It is shown pairs that should be close, such as two augmented views of the same image, and pairs that should be far apart, and it adjusts the embeddings so that related items cluster and unrelated items separate. Crucially, it needs no manual labels, since the notion of similar and dissimilar is derived from the data itself.

When paired across modalities, for example matching images to their captions, it produces embeddings where related images and text land near each other.

Positive pairs are constructed automatically, often as two augmentations of the same sample, and negatives are other samples in the batch. A contrastive loss increases the similarity of positives and decreases it for negatives, shaping an embedding space where distance reflects semantic similarity. Cross-modal variants align two different data types, such as images and text, in a shared space.

Contrastive learning is a major reason modern models can be trained on huge unlabeled datasets and still produce embeddings useful for search, clustering, and zero-shot tasks. Those embeddings are the backbone of semantic search and retrieval over large visual datasets.

The supervision signal, which pairs are similar, comes from the data itself, such as augmentations of the same image, rather than from human labels.

A positive pair should be close in the embedding space, like two views of one image. A negative pair should be far apart, like two unrelated images.

Self-supervised learning, Embedding and semantic search, Image Embeddings, Vision-Language Model (VLM), Fine-tuning.