ML3: Classic models with scikit-learn
Decision trees
🎯You'll be able to
Train a decision tree, read its rules, and explain how depth controls what it can learn.
🏠The everyday picture
A doctor's checklist: "Fever? If yes, is it above 102? If yes, any rash?" Each answer leads to the next question. A decision tree learns such a checklist from data.
📘What it really is
-
A decision tree splits the data with yes/no questions on one input at a time, choosing at each step the question that best separates the classes. Each final box, a leaf, gives a prediction: the share of late parcels in that box.
-
Depth is the number of questions in a row. A shallow tree learns broad rules; a deep tree can carve out tiny groups, including noise.
-
Strengths: handles combinations naturally, needs no scaling, and a shallow tree is easy to read. Limits: a single tree is unstable (a little more data can change its shape) and deep trees overfit badly.
-
Controls:
max_depth,min_samples_leaf(smallest box allowed) andmax_leaf_nodes.
📝A worked example
Input: decision trees of different depths on the kit (reference measurements, 2 November 2026, random_state=0):
| Depth | Training PR-AUC | Validation PR-AUC (May) |
|---|---|---|
| 2 | 0.4671 | 0.4184 |
| 4 | 0.597 | 0.4962 |
| 8 | 0.7937 | 0.3976 |
| No limit | 1.0 | 0.2307 |
The depth-2 tree's first question is "address score at most 0.33?", then "at most 0.20?". With no limit the tree memorises the training rows (1.0) and does worst on May. Depth 4 is the best of these.
Print a tree's rules with sklearn.tree.export_text(tree, feature_names=...).
🌍Where it's used
- Explanations: a shallow tree can summarise what a more complex model does.
- Operations: simple triage rules learned from data, then reviewed by experts.
- Building block: forests and gradient boosting are made of many trees (Lesson 3).
📝Case scenario
A call centre trained a deep tree to route calls and was pleased with its training accuracy. A month later the routing looked random. The tree had learned rules like "calls at 10:42 from this exchange go to billing". Limiting depth and requiring at least 50 calls per leaf gave simpler rules that held up.
🖱️Try it
Train a depth-3 tree and print its rules. Success: you can read the three questions on the path to the riskiest leaf, and they make sense for parcels (address score, zone, slot or similar).
⚠️Common mistakes
- No depth limit. Sign: training score 1.0, validation much lower. Fix: limit depth or leaf size.
- Trusting one tree's exact shape. Sign: a small data change gives a different tree. Fix: use ensembles for prediction (Lesson 3).
- Reading leaves as certainties. Sign: "this leaf is 0% late". Fix: check how many rows the leaf holds.
🛠️For your project
Project 1's chosen model is made of trees. A shallow tree is a good way to explain to the hub team what drives the flags.
✅Quick check
What happens to a decision tree with no depth limit?
Show the answer
It memorises the training rows and does worse on new data.
What does a leaf predict?
Show the answer
The share of each class among the training rows that ended up in it.