Confusion matrix
A confusion matrix is a table that breaks down a classification model's predictions by comparing predicted classes against true classes, showing where the model is right and exactly which classes it confuses. It is the basis for accuracy, precision, recall, and per-class error analysis.
In this article
Building visual or physical AI?
Let's talk.
What is a confusion matrix?
A confusion matrix is a grid with one axis for the true class and one for the predicted class. The diagonal holds the correct predictions, and every off-diagonal cell tells you how often one class was mistaken for another. For a binary problem it reduces to four cells: true positives, false positives, false negatives, and true negatives. Its value is that it does not collapse performance into one number, it shows the shape of a model's mistakes, and nearly every other classification metric is derived from it.
Key takeaways
- It is a table of true versus predicted classes, with correct predictions on the diagonal and confusions off it.
- For binary tasks it holds true positives, false positives, false negatives, and true negatives.
- Accuracy, precision, recall, and F1 are all computed from the matrix.
A confusion matrix is a snapshot at one threshold
Most classifiers output a score, and a threshold turns that score into a class. The confusion matrix you see shows the model at that one threshold. Lower it, and false negatives become true positives while false positives climb. Raise it, and the reverse happens. Neither version is wrong. The real question is which cell costs more.
In medical imaging the answer is usually clear. An abnormal scan called normal can mean a missed diagnosis, while a normal scan flagged for review costs a radiologist a few minutes. So the cell to study is the false negatives, and the useful question is which cases end up there. A normal versus abnormal model can look strong overall while consistently missing one finding that's subtle on an X-ray.
In Voxel51 the confusion matrix is interactive. Click a cell and the grid filters to the exact images behind it, with their ground truth, predictions, and other fields alongside. A count of false negatives becomes a list of specific scans you can read and group.
Chest X-Ray
10,000 images
NIH chest X-rays balanced across 14 findings, with radiologist boxes and an evaluated normal versus abnormal classifier.
Try this: Open the normal versus abnormal evaluation, click the false negative cell, and check which of the 14 findings those missed X-rays carry.
Open the X-rays in Voxel51
Free with a work or .edu email.
Go deeper
How to read it
- Fix your convention: typically rows are the true class and columns are the predicted class.
- A bright off-diagonal cell is a systematic confusion between two specific classes.
- Normalize each row to read per-class recall, how often each true class is correctly found.
Confusion matrix vs derived metrics
| Term | What it is |
|---|---|
| Confusion matrix | The full table of true versus predicted classes |
| Accuracy | The diagonal total over the grand total |
| Precision and recall | Column-wise and row-wise ratios pulled from the matrix |
Why it matters
If accuracy tells you how much a model gets wrong, the confusion matrix tells you what it gets wrong, and that is where the fixes are. Two patterns carry most of the signal. A bright off-diagonal cell means two classes are systematically confused, which is usually an ontology or labeling problem, ambiguous class boundaries, not a model problem, so the fix is in the schema and the labels. An entire dark row means a class the model almost never finds, often a rare or under-labeled class. Both point you to data and schema work rather than just more training.
For how a confusion matrix cell leads to the samples behind it, see sample-level review in model evaluation.
Frequently asked questions
What is a confusion matrix?
A table comparing a model's predicted classes against the true classes, showing correct predictions and specific confusions.
How do you read a confusion matrix?
The diagonal is correct predictions, off-diagonal cells show which classes are mistaken for which, and normalizing rows gives per-class recall.
What metrics come from a confusion matrix?
Accuracy, precision, recall, and F1 are all derived from its cells.
Related terms
Learn more
Explore the full evaluation workflow on the model evaluation page.
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 5, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved