ML2: Preparing data without fooling yourself
Pipelines: the same steps in training and in use
🎯You'll be able to
Put preparation and the model into one scikit-learn Pipeline, fit it on training data only, and save it as one object.
🏠The everyday picture
A restaurant that writes its recipe down, with exact amounts, gets the same dish whoever cooks. If each cook measures "a pinch" their own way, the dish changes. A pipeline is the written recipe: every step, in order, saved together.
📘What it really is
-
A
Pipelinechains steps: preparation (encoding, scaling) and then the model. Callingfitfits each step in turn; callingpredict_probaruns the same steps on new data. -
A
ColumnTransformerapplies different steps to different columns: one-hot for categories, scaling for numbers. -
Fit on training rows only. The scaler's mean, the encoder's categories and anything else learned from data must come from training rows. Fitting them on all rows leaks a little of the test set into training.
-
Save the whole pipeline, not just the model. The kit saves a bundle with
joblib.dump: the pipeline, the threshold, the version and the feature list. The API loads the same bundle. -
Share the feature code. The kit's
build_framemakes the booking hour and weekday in India time for both training and the API. If the API computed the hour differently, the model would get different inputs from the ones it learned on: training/serving skew (ML8 Lesson 3).
📝A worked example
Input:
from sklearn.pipeline import Pipeline
from sklearn.ensemble import HistGradientBoostingClassifier
from dakdost.train import encoder, split_by_time, load
from dakdost.features import build_frame
train, valid, test = split_by_time(load())
model = Pipeline([('prep', encoder(False)),
('clf', HistGradientBoostingClassifier(max_iter=200, learning_rate=0.05, max_leaf_nodes=15,
l2_regularization=1.0, random_state=0))])
model.fit(build_frame(train), train['late'])
print(model.predict_proba(build_frame(valid))[:3, 1].round(4))
Expected output: three probabilities between 0 and 1 for the first three May parcels. Everything the model learned, including the encoder's categories, came from January to April only.
🌍Where it's used
- Any deployed model: saving preparation and model together is the simplest defence against skew.
- Teams: a pipeline makes a model reviewable: every step is in code.
- Other tools: TensorFlow and PyTorch projects use the same idea with preprocessing layers or shared transform code.
📝Case scenario
A lending app's model worked in testing but rejected many good applicants in the app. The data team had scaled income in rupees per month; the app sent income per year. The model saw incomes twelve times larger than any it had learned from. Saving the scaler in the pipeline, and validating the input's unit, fixed it.
🖱️Try it
Train the pipeline above, save it with joblib.dump(model, 'models/try.joblib'), load it in a new Python session and predict the same
three May parcels. Success: identical probabilities.
⚠️Common mistakes
- Fitting preparation on all data. Sign: the scaler saw the test rows. Fix:
fitthe pipeline on training rows only. - Saving only the model. Sign: the API rebuilds preparation by hand. Fix: save the pipeline with its preparation.
- Two copies of feature code. Sign: the API and training compute a feature differently. Fix: one shared function, used by both.
🛠️For your project
Project 2's test 5 checks that the API's probability equals the saved pipeline's for the same parcel at four booking times.
✅Quick check
Why save the whole pipeline instead of just the model?
Show the answer
So that new data gets exactly the same preparation as the training data.
What is training/serving skew?
Show the answer
The model receiving inputs prepared differently in use from how they were prepared in training.