The confusion matrix, precision and recall
🎯You'll be able to
Build a confusion matrix, compute precision and recall, and explain them in terms of the people affected.
🏠The everyday picture
A fishing net that catches every fish also catches plastic and weeds. A net with fine selection catches only fish but lets many escape. Recall is "how many of the fish did we catch?"; precision is "how much of the catch is fish?".
📘What it really is
- The confusion matrix counts four outcomes for a yes/no prediction:
- true positives (TP): flagged and really late;
- false positives (FP): flagged but on time (an unnecessary call);
- false negatives (FN): not flagged but late (a missed parcel);
-
true negatives (TN): not flagged and on time.
-
Recall = TP ÷ (TP + FN): the share of late parcels that were flagged. Also called sensitivity, or the true positive rate.
- Precision = TP ÷ (TP + FP): the share of flagged parcels that really were late.
- The trade-off: flagging more parcels raises recall and usually lowers precision.
-
F1 combines the two into one number (their harmonic mean). It's convenient but hides the trade-off, and it treats both mistakes as equally costly.
-
Say it in people's terms: "of every 10 calls the hub makes, about 7 are to customers whose parcel would have been late" is clearer than "precision 0.66".
📝A worked example
Input: the kit's model on June at threshold 0.3 (reference figures, 2 November 2026).
from sklearn.metrics import confusion_matrix
tn, fp, fn, tp = confusion_matrix(y_june, p_june >= 0.3).ravel()
Expected output: TN 1,948, FP 58, FN 100, TP 115. So recall = 115 ÷ 215 = 0.5349 and precision = 115 ÷ 173 = 0.6647. In words: the hub makes 173 calls, about two in three reach a customer whose parcel would have been late, and about half of all late parcels are caught.
🌍Where it's used
- Health screening: recall matters most, so few real cases are missed; positives are then checked by a doctor.
- Spam filters: precision matters most, so real emails aren't hidden.
- Content moderation: both matter, with people reviewing borderline cases.
📝Case scenario
A city's model flagged buildings for fire inspection. Inspectors complained that most visits found nothing: precision was low. Raising the threshold cut wasted visits, but the fire service noticed recall fell too, so a few risky buildings were no longer visited. They agreed a minimum recall first, then raised precision as far as possible within it, which is the rule the kit uses.
🖱️Try it
Compute the confusion matrix for threshold 0.15 on June. Success: TP 142, FP 100, FN 73, TN 1,906, giving recall about 0.66 and precision about 0.59.
⚠️Common mistakes
-
Mixing up precision and recall. Sign: "recall is how many flags were right". Fix: recall is about the late parcels; precision is about the flags.
-
Reporting one without the other. Sign: "recall 0.9!" with precision 0.1. Fix: report both, and the threshold.
- Leaning on F1 by habit. Sign: F1 chosen without asking what mistakes cost. Fix: use the costs (Lesson 4).
🛠️For your project
Project 1 reports precision, recall and the number flagged, at the chosen threshold, for May, June and each destination zone.
✅Quick check
What is recall in DakDost's terms?
Show the answer
The share of late parcels that the model flagged.
What does a false positive cost DakDost?
Show the answer
An unnecessary phone call to a customer whose parcel would have been on time.