Level 6 · Cloud Computing, DevOps and Cybersecurity · Course 6.3 DevOps and Operations

Module 72: Production Operations

Goal: keep your CRM healthy once real people use it: see what it's doing through logs and metrics, get told when something is wrong, agree what "up" means, prove you can recover data, and handle incidents calmly.

You'll need: your deployed CRM on staging (and production if ready) from Modules 70 and 71.

Lessons

  1. 1Logs and metrics15–25 minutes✓
  2. 2Alerts15–25 minutes✓
  3. 3Uptime targets15–25 minutes✓
  4. 4Backup and recovery15–25 minutes✓
  5. 5Incident response15–25 minutes✓

🛠️Module practice: Project C, part 2, an operated application

  1. Structured logs with request IDs, no secrets in logs, and log rotation.
  2. A dashboard (or provider view) of the four golden signals.
  3. Outside uptime monitoring and at least three tested alerts, each with a short runbook.
  4. Written SLOs with error budgets.
  5. Automated, encrypted, off-site backups with failure alerts and a timed restore drill.
  6. An incident checklist, one practised drill and its blameless review, all in your operations guide.

✅Module check

Explain in a few sentences, in your own words:

How do you keep a production application healthy and recover when things go wrong?

Show a good answer

A good answer mentions: structured logs (with request IDs, rotation and no secrets) and metrics on the four golden signals; a few actionable, tested alerts on user-facing symptoms, checked from outside and linked to runbooks; SLOs with error budgets chosen by need; automatic, encrypted, off-site, monitored backups following 3-2-1, proven by timed restore drills against RPO and RTO; and an incident process that mitigates first (often by rollback), communicates clearly, keeps a timeline and ends with a blameless review and action items.

📝Module quiz

10 questions · 10 marks each · 100 marks in all · pass mark 70% · retry as often as you like ·

1. Which are the four golden signals?

Why each option is right or wrong

  • “Availability, durability, consistency and cost” (Incorrect): Useful qualities, but the golden signals are latency, traffic, errors and saturation.
  • “CPU, memory, disk and network usage only” (Incorrect): Those are resource measures; the golden signals also include latency, traffic and errors.
  • “Uptime, downtime, deploys and rollbacks” (Incorrect): These are events and summaries, not the four golden signals.
  • “Latency, traffic, errors and saturation” is correct.

    These four describe a service's health from the user's side: how long requests take, how many arrive, how many fail, and how close resources are to their limit. Watching them catches most problems.

    • Latency: watch slow percentiles like p95, not just the average.
    • Saturation: CPU, memory, disk, connections.

📖 Revise: Lesson 1: Logs and metrics

Question ID module-72-q1 · Report a mistake in this question
2. Why add a request ID to every log line?

Why each option is right or wrong

  • “It makes log files smaller and faster to search” (Incorrect): It adds a little to each line; the benefit is tracing.
  • “It counts how many requests the server has handled” (Incorrect): Metrics count requests; the ID links lines together.
  • “It's required before any log line can be written to the disk” (Incorrect): Logs work without it; tracing is just harder.
  • “To follow everything that happened during one request” is correct.

    A busy server interleaves thousands of requests in its logs. A shared ID on every line from one request, also returned to the user, lets you filter to exactly that request's story when someone reports a problem.

    • Pass it on: to other services in a header.
    • Show it: in error messages as a reference number.

📖 Revise: Lesson 1: Logs and metrics

Question ID module-72-q2 · Report a mistake in this question
3. What's alert fatigue?

Why each option is right or wrong

  • “Tiredness from working long shifts on call” (Incorrect): Shifts matter, but alert fatigue is about too many alerts.
  • “An alert that fires when the server is overloaded” (Incorrect): That's a saturation alert; fatigue is ignoring alerts.
  • “A monitoring tool slowing down after collecting too much data” (Incorrect): It's a human problem, not a tool problem.
  • “So many alerts that people ignore them, including important ones” is correct.

    When alerts fire constantly for things that don't need action, people mute or ignore them. Then the real emergency is missed. Fewer, actionable alerts keep everyone responsive.

    • Each alert should: need a human action now.
    • Everything else: dashboards or daily summaries.

📖 Revise: Lesson 2: Alerts

Question ID module-72-q3 · Report a mistake in this question
4. Why check uptime from outside your server?

Why each option is right or wrong

  • “Outside checks make the website load faster for real visitors” (Incorrect): Checks don't affect speed for users.
  • “Servers can't run monitoring software” (Incorrect): They can, but they can't report their own outage.
  • “It's cheaper than any monitoring on the server” (Incorrect): Cost isn't the main reason; independence is.
  • “A server that's down can't report that it's down” is correct.

    If monitoring runs only on the server, a crash, network problem or power cut silences it too. An outside service checking your site every minute notices when nobody can reach it.

    • Check from: several locations.
    • Check: a real page or health endpoint, not just a ping.

📖 Revise: Lesson 3: Uptime targets

Question ID module-72-q4 · Report a mistake in this question
5. What's an error budget?

Why each option is right or wrong

  • “Money set aside to pay for server errors” (Incorrect): It's an allowance of failure, measured in time or requests.
  • “The number of known bugs allowed in the code at any time” (Incorrect): It's measured in user-facing failures, not bugs counted.
  • “A discount customers receive automatically after each outage” (Incorrect): That's an SLA credit, not an error budget.
  • “The failure an SLO allows; when spent, focus on reliability” is correct.

    A 99.9% availability target allows about 43 minutes of downtime a month. That allowance is the budget. While some is left, the team ships features; once it's used up, they prioritise reliability work.

    • Turns reliability: into a shared, measurable decision.
    • Avoids: arguing about whether to slow down.

📖 Revise: Lesson 3: Uptime targets

Question ID module-72-q5 · Report a mistake in this question
6. What's the difference between an SLO and an SLA?

Why each option is right or wrong

  • “They're the same thing, just written by different teams” (Incorrect): One is internal, the other contractual.
  • “An SLA is internal; an SLO is a promise to customers” (Incorrect): That's the wrong way round.
  • “An SLO is for websites; an SLA is for databases” (Incorrect): Both apply to any service.
  • “An SLO is an internal target; an SLA is a promise to customers” is correct.

    A service level objective is the target your team aims for, like 99.9% uptime. A service level agreement is a contract with customers, often with penalties if missed, so it's usually set a little lower than the SLO.

    • SLI: the actual measurement, like successful requests.
    • Set SLA below SLO: to leave a safety margin.

📖 Revise: Lesson 3: Uptime targets

Question ID module-72-q6 · Report a mistake in this question
7. What's the 3-2-1 backup rule?

Why each option is right or wrong

  • “Back up three times a day, for two weeks, in one place” (Incorrect): Frequency and retention matter, but the rule is about copies and locations.
  • “Three days of backups, two people with access, one restore test a year” (Incorrect): Retention and access matter, but the rule is about copies, storage types and location.
  • “Keep three servers, two databases and one website” (Incorrect): It's about backup copies, not servers.
  • “Three copies, two kinds of storage, one off-site” is correct.

    Keep the live data plus two backups, on at least two different kinds of storage, with at least one copy somewhere else. One failure, theft or fire can't then destroy every copy.

    • Add: one copy that can't be changed or deleted, against ransomware.
    • Test: restores regularly.

📖 Revise: Lesson 4: Backup and recovery

Question ID module-72-q7 · Report a mistake in this question
8. How do you know a backup works?

Why each option is right or wrong

  • “The backup job reports success” (Incorrect): Jobs can report success while the backup is incomplete or broken.
  • “The backup file is larger than zero bytes” (Incorrect): A non-empty file can still be unusable.
  • “The provider says backups are enabled” (Incorrect): Enabled doesn't prove you can restore.
  • “By restoring it and checking, on a schedule” is correct.

    Backup jobs can succeed while producing unusable files. Restoring into a separate place, checking the data and timing it is the only proof, and doing it regularly keeps the steps familiar.

    • Check: row counts and recent records.
    • Write down: the steps and how long they took.

📖 Revise: Lesson 4: Backup and recovery

Question ID module-72-q8 · Report a mistake in this question
9. In an incident, what comes first?

Why each option is right or wrong

  • “Finding who caused the problem” (Incorrect): Blame doesn't help users; restoring service does.
  • “Writing the post-incident report” (Incorrect): The report comes after the incident is resolved.
  • “Fixing the root cause perfectly first” (Incorrect): A perfect fix can take hours; restore service first.
  • “Restoring service” is correct.

    Users are affected now. First stop the damage: roll back, fail over, or switch off the broken feature. Investigating the root cause comes after service is back.

    • Communicate: a short status update early.
    • Write notes: of what you did and when, for the review.

📖 Revise: Lesson 5: Incident response

Question ID module-72-q9 · Report a mistake in this question
10. Why should post-incident reviews be blameless?

Why each option is right or wrong

  • “So nobody has to write anything down after an incident” (Incorrect): Reviews still produce written findings and actions.
  • “Because incidents are nobody's responsibility” (Incorrect): People are responsible; blame just stops honest learning.
  • “So the incident can be kept secret from the rest of the company” (Incorrect): Blameless reviews are shared to help everyone learn.
  • “So people share what really happened and the system gets fixed” is correct.

    If people expect blame, they hide mistakes, and the real causes, like missing checks or confusing tools, stay unfixed. Focusing on how the system allowed the problem leads to lasting improvements.

    • Ask: what made this mistake easy to make?
    • Output: concrete actions with owners.

📖 Revise: Lesson 5: Incident response

Question ID module-72-q10 · Report a mistake in this question

ProLearners Assist