CD10: Capstone: three DevOps projects for RouteRani
Project 2: a pipeline with a way back
🎯You'll be able to
Make the workflow safe and the pipeline script check and roll back releases, until all 10 pipeline tests pass.
🏠The everyday picture
A medicine factory doesn't send a batch out because the machines ran. It tests samples from the batch, and if they fail, the batch is recalled and the previous batch stays on shelves. The process runs the same way whoever is on shift.
📘What it really is
Project 2 asks you to fix:
-
the workflow (CD7): runs on every push to any branch,
write-allpermissions, actions on@main, a deploy with noneedsthat runs from any branch, a step that prints a secret, no concurrency. Fix:pull_requestpluspushtomain,contents: read, pinned actions,needs: test, anif:formain,environment: production, a concurrency group, timeouts, and the test job's database service, Terraform checks and alert-rule tests; -
the pipeline script (CD8): it builds and deploys only. Add: a version check before anything else, a test stage that stops before building if a test fails, verification under traffic (failures under 1%, p95 under 500 ms, every copy on the new version), automatic rollback to the previous version read from Terraform's output, recording the live version in
infra/release.auto.tfvars, and a non-zero exit after any rollback; -
Terraform: add
release_fault(practice only) so the tests can deploy bad releases.
📝A worked example
Input: after your changes, run npm run test:pipeline.
Expected output: # pass 10. Tests 7 to 9 each deploy a bad release and expect 1.0.2 to be live afterwards with no bad copies left; test 10
adds a failing API test and expects no image to be built. On the test machine on 1 November 2026, the reference solution kept every request
working during a good deploy under load (46,478 requests, 0 failed), rolled back an unhealthy release after about 35 seconds, and caught the
failing and slow releases at 25.2% failures and a p95 of 1,807 ms.
🌍Where it's used
- Release engineering in any team that deploys daily.
- Pipeline security reviews after supply-chain incidents.
- On-call: an automatic rollback at 2 am is a page that doesn't need to happen.
📝Case scenario
A learner's test 10 passes on its own but fails in the full run: the failing test they add doesn't stop the pipeline. The cause is that the
pipeline is started from inside a Node.js test, which sets an environment variable (NODE_TEST_CONTEXT) that changes how the inner npm test
reports. Removing that variable from the test stage's environment made the inner tests report normally. Reading the pipeline's own output,
not only the test's message, found it.
🖱️Try it
Run the three practice faults by hand before running the tests: node tools/pipeline.js 1.0.3 --fault=unhealthy --skip-tests and so on, with
1.0.2 live. Success: each run rolls back and exits with code 1, and curl localhost:8080/version shows 1.0.2 each time.
⚠️Common mistakes
- Verifying before the old copies are gone. Sign: verify passes while traffic still reaches old copies. Fix: check every response's version.
- Reading "previous" after deploying. Sign: rollback "returns" to the bad version. Fix: read the live version before applying the new one.
- Rollback that exits 0. Sign: green pipelines after failed releases. Fix: exit with an error after any rollback.
🛠️For your project
Save the pipeline's output for a good deploy under load and for each of the three faults on the Project 2 page.
✅Quick check
Why must the pipeline read the previous version before deploying?
Show the answer
After the deploy, Terraform's output already shows the new (possibly bad) version.
What should happen when a test fails in the test stage?
Show the answer
The pipeline stops before building or deploying anything.