Confusion Matrix in Machine Learning with EXAMPLE

What is Confusion Matrix?

A confusion matrix is a performance measurement technique for machine learning classification. It is a table that shows how a classification model performed on a set of test data for which the true values are already known. The term confusion matrix is simple enough, but the terminology built on top of it can be confusing, so each piece is explained below in plain language.

The matrix applies to any supervised classifier — logistic regression, a decision tree, a Naive Bayes model or a deep neural network — because it only compares two columns of labels: what the model predicted and what was actually true.

Four outcomes of the confusion matrix

The confusion matrix visualizes the accuracy of a classifier by comparing the actual and predicted classes. The binary confusion matrix is composed of squares:

The table above maps the four squares that every binary confusion matrix contains:

• TP: True Positive: Predicted values correctly predicted as actual positive
• FP: False Positive: Predicted values incorrectly predicted as actual positive, i.e., negative values predicted as positive
• FN: False Negative: Positive values predicted as negative
• TN: True Negative: Predicted values correctly predicted as an actual negative

Statistics gives the two error cells their own names. A false positive is a Type I error — the model raised an alarm that should never have been raised. A false negative is a Type II error — the model stayed silent when it should have raised the alarm. Knowing which of the two is more expensive for your problem decides which metric you tune later.

You can compute the accuracy test from the confusion matrix, as the formula below shows:

Example of Confusion Matrix

Confusion Matrix is a useful machine learning method which allows you to measure Recall, Precision, Accuracy, and the AUC-ROC curve. The football example below shows what the terms True Positive, True Negative, False Positive and False Negative mean in everyday language.

True Positive:

You predicted positive and it turned out to be true. For example, you had predicted that France would win the world cup, and it won.

True Negative:

You predicted negative, and that is also true. You had predicted that England would not win, and it lost.

False Positive:

Your prediction is positive, and it is false.

You had predicted that England would win, but it lost.

False Negative:

Your prediction is negative, and the result makes it false.

You had predicted that France would not win, but it won.

You should remember that the first word describes whether the prediction was right or wrong (True or False) and the second word describes what the model predicted (Positive or Negative).

How to Calculate a Confusion Matrix

Here is the step by step process for calculating a confusion matrix in data mining:

• Step 1) First, you need a test dataset together with its expected outcome values.
• Step 2) Predict all the rows in the test dataset.
• Step 3) Compare the expected outcomes with the predictions and count:
  1. The total of correct predictions of each class.
  2. The total of incorrect predictions of each class.

After that, these numbers are organized in the below-given methods:

• Every row of the matrix corresponds with an actual class.
• Every column of the matrix links to a predicted class.
• The total counts of correct and incorrect classification are entered into the table.
• The sum of correct predictions for a class goes into the cell where that class’s actual row meets its own predicted column — the diagonal.
• The sum of incorrect predictions for a class goes into the actual row for that class value and the predicted column of whichever class the model chose instead.

Row and column roles are a convention rather than a law, and some plotting tools transpose the layout, so always read the axis labels before interpreting a matrix. The orientation used here — actual on the rows, predicted on the columns — is the one scikit-learn produces.

Other Important Terms using a Confusion matrix

Once the four counts are in place, a family of secondary terms describes different slices of the same table:

• Positive Predictive Value (PPV): This is very close to precision. One significant difference between the two terms is that PPV takes prevalence into account. In a situation where the classes are perfectly balanced, the positive predictive value is the same as precision.
• Null Error Rate: This term defines how often your prediction would be wrong if you always predicted the majority class. You can treat it as a baseline metric to compare your classifier against.
• F Score: The F1 score is a weighted average score of the true positive rate (recall) and precision.
• ROC Curve: The ROC curve plots the true positive rate against the false positive rate at various cut points. It also demonstrates a trade-off between sensitivity (recall) and specificity, which is the true negative rate.
• Precision: The precision metric shows the accuracy of the positive class. It measures how likely the prediction of the positive class is correct.

The maximum score is 1 when the classifier perfectly classifies all the positive values. Precision alone is not very helpful because it ignores the negative class. The metric is usually paired with the recall metric. Recall is also called sensitivity or true positive rate, and it is written as shown below.

• Sensitivity: Sensitivity computes the ratio of positive classes correctly detected. This metric shows how good the model is at recognizing a positive class.

Confusion Matrix Metrics and Formulas

Every metric above is arithmetic on the same four counts, so it helps to see them side by side with the question each one answers.

Take a spam filter tested on 100 emails that produces TP = 45, FN = 5, FP = 10 and TN = 40. Accuracy is (45 + 40) / 100 = 0.85. Precision is 45 / (45 + 10) = 0.82, recall is 45 / (45 + 5) = 0.90, and specificity is 40 / (40 + 10) = 0.80. The F1 score works out to 0.86.

Those numbers tell a story that a single accuracy figure hides: the filter catches 90 percent of real spam but wrongly quarantines one legitimate email in five flagged. Whether that trade is acceptable depends on the cost of each error, which is precisely why the matrix is reported instead of accuracy alone.

Confusion Matrix for Multi-Class Classification

Classification problems rarely stop at two labels, and the matrix scales without changing shape. For N classes the table becomes an N×N grid: the diagonal holds every correct prediction, and each off-diagonal cell records exactly which class was confused for which other class.

A three-class model that sorts images into cat, dog and rabbit produces a 3×3 grid. If the cell at row “cat”, column “dog” holds 12, then twelve cat images were labelled dog. That level of detail is why the matrix is more useful than a score: it names the specific pair of classes the model cannot separate.

Precision, recall and F1 are defined per class using a one-vs-rest view, where the class in question is the positive class and everything else is negative. The per-class figures are then combined in one of three ways:

• Macro average: Computes the metric for each class independently, then takes the unweighted mean. Every class counts equally, so rare classes are not drowned out.
• Micro average: Pools the TP, FP and FN counts across all classes before computing the metric. Large classes dominate, and for single-label problems micro precision, micro recall and accuracy are identical.
• Weighted average: Averages the per-class scores using the number of true instances of each class as the weight, which keeps class imbalance visible.

Choose macro when every class matters equally, and weighted when the class distribution reflects real traffic.

How to Create a Confusion Matrix in Python

The scikit-learn library builds the whole table from two label arrays, so no manual counting is required. The example below compares ten true labels against ten predictions.

from sklearn.metrics import confusion_matrix

y_true = [1, 0, 1, 1, 0, 1, 0, 0, 1, 0]
y_pred = [1, 0, 1, 0, 0, 1, 1, 0, 1, 0]

cm = confusion_matrix(y_true, y_pred)
print(cm)

The call returns a 2×2 NumPy array in which row 0 is the actual negative class and row 1 is the actual positive class:

[[4 1]
 [1 4]]

Reading the array against the scikit-learn convention gives TN = 4 (top left), FP = 1 (top right), FN = 1 (bottom left) and TP = 4 (bottom right). Unpacking those four values in one line makes the mapping explicit:

tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()

To get precision, recall and F1 for every class at once, including the macro and weighted averages described above, call classification_report() instead of computing each metric by hand:

from sklearn.metrics import classification_report

print(classification_report(y_true, y_pred))

For a plotted version, ConfusionMatrixDisplay.from_predictions(y_true, y_pred) renders the same table as a labelled heatmap. The full argument list, including the labels and normalize options, is documented in the scikit-learn confusion_matrix reference. The same evaluation step applies to models built with TensorFlow, because the metric depends only on the predicted labels.

Why you need Confusion matrix?

Here are the pros and benefits of using a confusion matrix.

• It shows how a classification model becomes confused when it makes predictions.
• The confusion matrix gives you insight not only into the errors your classifier makes, but also into the types of errors it makes.
• This breakdown helps you overcome the limitation of using classification accuracy alone.
• Every column of the confusion matrix represents the instances of that predicted class.
• Each row of the confusion matrix represents the instances of the actual class.
• It turns model evaluation into a diagnosis, pointing at the specific class pair that needs more data or a better feature.

That diagnostic value is why the confusion matrix sits at the centre of the evaluation stage in any data science workflow, and why it is usually the first table reviewed before a classifier is promoted to production.

FAQs