Cloud / DevOps Engineer · Projects
Project 2: A pipeline with a way back
Marking guide version 1 · Lessons: CD1 lesson 5, CD7 lessons 1–5, CD8 lessons 1–4, CD10 lesson 3
The problem and who it's for
RouteRani's workflow deploys from any branch, with a token that can do anything, actions that change without warning, and a step that prints a secret. The deploy script accepts any version, never runs the tests and has no way back: a bad release stays live until someone notices. On a Friday evening, when bookings for the weekend peak, a release that fails a quarter of requests would cost hundreds of bookings.
Scope
- Run the workflow on pull requests and on pushes to
mainonly, with a read-only token and pinned actions. - Remove the secret-printing step; make deploy need the tests, run only from
main, use theproductionenvironment and a concurrency group. - Give the test job a database service, the API tests, Terraform format and validate, and the alert-rule tests.
- Make
tools/pipeline.jsrefuse versions that aren't x.y.z, run the tests first, verify each release under traffic, and roll back by itself. - Add the practice-only
release_faultvariable to Terraform so bad releases can be simulated.
Not included
- Running the workflow on GitHub (optional; the tests check the file on your computer).
- A real cloud deploy or OIDC role (CD6 and CD7 design notes cover them).
- Canary or blue-green deploys (discuss them in your write-up if you like).
Before you start
- Core: Module 10 (Working with GitHub), Module 71 (CI/CD), Module 72 (Production Operations).
- Path: CD7 and CD8 complete, and Project 1 finished (the pipeline needs its variables, outputs and start-before-stop copies).
npm run test:pipelinerunning (most of its 10 tests fail at first).
Starter files
routerani/.github/workflows/ci.yml: runs on every push,write-all, actions on@main, a deploy with noneeds, a step that prints a secret.routerani/tools/pipeline.js: builds and deploys only.routerani/tools/load.js: the load tool (finished).routerani/tests/pipeline.test.js: the 10 acceptance tests for this project.
Download everything at once: starter/cloud-devops-starter.zip.
Milestones
- Run
npm run test:pipelineand note which of the 10 tests fail and why. - Fix the workflow's triggers, permissions, pinning and secret step (tests 1 to 3).
- Add
needs, themaincondition, the environment and concurrency (test 4). - Add the version check and the test stage (tests 5 and 10).
- Add verification and rollback, and the
release_faultvariable (tests 6 to 9). - Run all 10 tests twice in a row, and record a deploy under load and the three practice faults.
Expected outputs
- 10 of 10 pipeline tests passing, twice in a row.
- A deploy under load with 0 failed requests, recorded with the load tool's numbers.
- Three rollbacks recorded (unhealthy, errors, slow), each ending with the previous version live.
Acceptance tests
| # | Input | Expected result |
|---|---|---|
| 1 | npm run test:pipeline |
10 of 10 pass, twice in a row |
| 2 | node tools/pipeline.js latest |
Stops with an error before building anything (test 5) |
| 3 | node tools/pipeline.js 1.0.2 --skip-tests while tools/load.js sends traffic |
1.0.2 live; at most 0.5% of requests failed (test 6) |
| 4 | node tools/pipeline.js 1.0.3 --fault=unhealthy --skip-tests |
Non-zero exit; 1.0.2 live; no unhealthy copies left (test 7) |
| 5 | The same with --fault=errors and --fault=slow |
Each rolled back by itself, with the reason in the output (tests 8 and 9) |
| 6 | A failing test added to app/test/, then node tools/pipeline.js 1.0.6 |
Stops before an image is built (test 10) |
Failure and recovery exercises
- Change the verify limit from 1% to 30% and run the
errorsfault. Explain what happened and why the limit matters, then restore it. - Read the previous version after the deploy instead of before, and run the
unhealthyfault. Describe what the rollback does, then fix it. - In the workflow, put
${{ github.event.pull_request.title }}inside arun:line, and explain in two sentences how an attacker could use it. Then change it to pass the title throughenv:.
Evidence to submit
- The deliverable link to your repository or folder with the workflow and pipeline changes.
- The output of
npm run test:pipelineshowing 10 passed, pasted into test evidence. - The pipeline's output for a deploy under load and for each practice fault.
- Your explanation: what each job does, when deploy runs, how a release is checked, and how the rollback finds the previous version.
Marking guide and self-check
Pass rule: every Safety-critical (S) and Mandatory (M) item must be met. A single unmet safety item means the project is not complete, whatever else is done. Quality items are for your own improvement.
Your answers are saved in this browser on this device. Self-checked means you checked your own work against this guide; it is not a reviewer's approval. Never paste passwords, keys, tokens or real customer data here.
