ML1: Thinking in machine learning
Baselines first
🎯You'll be able to
Measure a "do nothing" baseline and a simple rule, so you know what a model has to beat.
🏠The everyday picture
Before buying an expensive water filter, you'd check how clean the tap water already is. A baseline is the tap water: the result you get with no model at all, or with the simplest rule a sensible person would try.
📘What it really is
- A baseline is the simplest honest answer. Two kinds are useful:
- The majority answer: always predict "on time". It's right whenever parcels are on time.
-
A simple rule: something a domain expert would try first, such as "flag every parcel whose address score is below 0.5".
-
Why it matters: a model that doesn't beat a simple rule adds cost and risk for nothing. A model that beats it by a little might still not be worth running.
-
Compare on the same data and with the same measures. For DakDost, use the June parcels (the test month in the kit) and measure:
- recall: the share of late parcels that were flagged;
-
precision: the share of flagged parcels that really were late. ML4 explains both in detail.
-
A baseline doesn't need to be clever. It needs to be fair, written down and reported next to the model.
📝A worked example
Input: the June parcels from the kit.
june = df[df['booked_at'].str.startswith('2026-06')]
print(len(june), round(june['late'].mean(), 4))
print('always on time, accuracy:', round(1 - june['late'].mean(), 4))
rule = june['address_quality'] < 0.5
caught = (rule & (june['late'] == 1)).sum()
print('rule flags', rule.sum(), 'recall', round(caught / june['late'].sum(), 4), 'precision', round(caught / rule.sum(), 4))
Expected output: 2221 0.0968; "always on time" is 90.32% accurate but catches no late parcels; the address rule flags 246 parcels,
with recall 0.6093 and precision 0.5325. So a model must do better than "catch 61% of late parcels while 53% of the calls are useful".
The kit's model, at its chosen threshold, has precision 0.6647 and recall 0.5349 on June: more useful calls, but it catches fewer late
parcels than the rule. Which is better depends on the cost of calls and of late parcels (ML4 Lesson 4).
🌍Where it's used
- Forecasting: "the same as last week" is the baseline every sales forecast must beat.
- Fraud: existing rules (amount limits, new device) are the baseline a fraud model must improve on.
- Search: plain keyword search is the baseline for a smarter ranking.
📝Case scenario
A food delivery company spent two months on a model to predict which orders would be cancelled. When someone finally compared it with the rule "flag orders from new customers paying cash at night", the rule caught nearly as many cancellations and was easier to explain. The team kept the rule and moved the model work to a question the rule couldn't answer.
🖱️Try it
Try the rule "address score below 0.5 or remote destination" on June. Success: it catches more late parcels (recall about 0.79) but precision falls to about 0.40, so more calls are wasted.
⚠️Common mistakes
- No baseline at all. Sign: a model's score can't be judged. Fix: report the majority answer and a simple rule next to it.
- A weak straw-man baseline. Sign: the baseline is something nobody would use. Fix: ask a domain expert what they'd try first.
- Comparing on different data. Sign: the baseline is scored on one month, the model on another. Fix: same rows, same measures.
🛠️For your project
Project 1's test 4 checks that metrics.json reports a baseline on June; test 5 checks that the model clearly beats it.
✅Quick check
Why is "always on time" 90% accurate in June but useless?
Show the answer
Because about 90% of parcels are on time anyway, and it catches no late ones.
What makes a good simple-rule baseline?
Show the answer
Something a sensible expert would try first, measured on the same data as the model.