Packaging and deploying a model service
🎯You'll be able to
Package the API and model so it runs the same everywhere, and plan a safe rollout and rollback.
🏠The everyday picture
A tiffin service packs each meal in a labelled box with the date and what's inside, so whoever delivers it knows exactly what they carry, and an old box can be swapped back if a new recipe disappoints.
📘What it really is
-
Package the code, the pinned libraries and the model file (or a way to fetch a specific model version). A Docker image (Module 69) is the usual package: same Python, same libraries, same model, everywhere.
-
Configuration from the environment: the kit's API reads
DAKDOST_MODELandDAKDOST_PREDICTION_LOG, so the same image can load a different model file without code changes. -
Health checks:
/healthreportsokand the model version, so a load balancer or orchestrator knows the service is ready (the Cloud / DevOps path covers this in depth). -
Rollout: release a new model to a small share of traffic first (canary), or score it silently next to the old one (shadow), compare, then switch. Keep the previous version ready to switch back.
-
Large models may need GPUs, a dedicated serving tool (such as NVIDIA Triton or a model server from a cloud provider) and more memory; the same principles apply. These aren't verified in this kit.
📝A worked example
Input: a plan for DakDost's API (not verified in this kit; the Cloud / DevOps path shows Docker images and health checks working).
Steps:
- A
Dockerfilebased on a slim Python 3.13 image installsrequirements.txt, copiesdakdost/and a model file, runs as an ordinary user, and startsuvicorn dakdost.serve:app --host 0.0.0.0 --port 8000. - The model file is named by version (
model-2026.07.1.joblib) and chosen withDAKDOST_MODEL. - Release: run the new version in shadow for a week, logging both versions' probabilities for the same requests; compare flag rates and, once outcomes arrive, precision and recall; then switch, keeping the old image tagged.
Expected outcome: any running copy can say which model it serves, a bad model can be replaced by changing one setting, and a release can be undone in minutes.
🌍Where it's used
- Product teams: model APIs in containers behind a load balancer.
- Platforms: managed model-serving services that host a saved model behind an endpoint.
- Edge devices: small models packaged into apps or devices (Mobile and IoT paths).
📝Case scenario
A team replaced their model by copying a new file onto the server, overwriting the old one. The new model had a bug, and the old file was gone. They rebuilt it from a week-old notebook, under pressure. Afterwards, every model file got a version in its name, was stored in a registry, and was selected by a setting.
🖱️Try it
Train two versions of the model with different settings and save them with versioned names. Start the API with DAKDOST_MODEL pointing at
each in turn and call /health. Success: the version in /health changes with the setting, with no code change.
⚠️Common mistakes
- Overwriting the model file. Sign: no way back. Fix: versioned files and a setting to choose one.
- Building images with whatever is installed. Sign: "works on my machine". Fix: pinned requirements in the image.
- Switching all traffic at once. Sign: a bad model affects everyone. Fix: shadow or canary first.
🛠️For your project
Project 2's test 1 checks /health reports the model version; the capstone design notes cover rollout and rollback.
✅Quick check
What does a shadow release do?
Show the answer
Scores real requests with the new model without using its answers, so the two versions can be compared.
Why choose the model file through a setting?
Show the answer
So a different version can be loaded, or rolled back, without changing code.