IoT and Smart Systems Engineer · Focus area 1: Embedded programming and firmware

IO4: Reliable firmware: interrupts, RTOS tasks, watchdogs and safe state

Goal: respond to events on time with interrupts, split work into RTOS tasks safely, recover from hangs with a watchdog, keep state safe through resets and power cuts, and diagnose faults on devices you can't reach.

You'll need: IO2, IO3, Module 2 (Operating Systems), Module 24 (Asynchronous Programming), Module 96 (Reliability Engineering), and the JalSetu practice kit (firmware/state.py, sim/flash.py, sim/harness.py).

Where this is used: any device that must keep working for months unattended: meters, alarms, medical monitors, factory controllers and vehicle electronics. Many run FreeRTOS or Zephyr; all need to recover from faults by themselves.

How it differs: a crashed web server is restarted by the platform and its logs are in the cloud. A crashed device must restart itself, keep its data safe, and leave enough clues for someone far away to understand what happened.

Time: reading about 3 hours · practice (the Try it tasks) about 3 hours · assignment about 4 hours.

Lessons

  1. 1Interrupts: reacting on timeReading 15–25 minutes, plus practice✓
  2. 2RTOS tasks, queues and prioritiesReading 15–25 minutes, plus practice✓
  3. 3Watchdogs and recovering by yourselfReading 15–25 minutes, plus practice✓
  4. 4Keeping state safe: RTC memory, flash and power cutsReading 15–25 minutes, plus practice✓
  5. 5Diagnosing faults in the fieldReading 15–25 minutes, plus practice✓

🛠️Assignment: JalSetu's reliability

  1. Make sensor faults handled errors, so every wake completes and sleeps.
  2. Save the state in RTC memory as JS1, the JSON and a CRC32, with safe defaults when it's missing or damaged.
  3. Sketch JalSetu as RTOS tasks with queues and a mutex for the I2C bus.
  4. Design JalSetu's status message for diagnosis.
  5. Run pytest tests/test_power.py and note which tests pass (3 and 6 at least).

Self-check before you finish:

✅Module check

Explain in a few sentences, in your own words:

How does firmware keep working for months without anyone touching it?

Show a good answer

A good answer mentions: short interrupt handlers that set flags or queue data, with debouncing; RTOS tasks with priorities, queues and short-held mutexes, avoiding priority inversion and deadlock; a watchdog fed only after real progress, the reset cause recorded and expected errors handled; state saved with a magic marker and CRC, flash appended or written then renamed, and writes batched to limit wear; and health information (version, battery, resets, errors, backlog) in each status, with last-heard times and small event logs.

📚Official references

Official documentation and guidance for this module's topics, chosen on 2 November 2026. Tools, laws and standards change, so check the current version before you rely on a detail. If a link has moved, search the same site.

📝Module quiz

10 questions · 10 marks each · 100 marks in all · pass mark 70% · retry as often as you like ·

1. An interrupt handler for a water meter's pulses also writes each count to flash. Pulses go missing during heavy use. What's the fix?

Why each option is right or wrong

  • “Write to flash twice in the handler to make sure each pulse is saved” (Incorrect): That makes the handler even slower.
  • “Turn off interrupts and check the pulse pin once a minute” (Incorrect): Pulses between checks would be missed.
  • “Use a larger flash chip so that the writes finish more quickly” (Incorrect): Size doesn't make writes fast enough for a handler.
  • “Count in the handler, and save totals to flash from the main loop” is correct.

    The handler stays short, so no pulses are missed.

    • Also: protect the shared count.

📖 Revise: Lesson 1: Interrupts: reacting on time

Question ID sp-iot-04-q1 · Report a mistake in this question
2. A device freezes once a week. Two RTOS tasks each hold one mutex and wait for the other's. What is this?

Why each option is right or wrong

  • “Priority inversion, which the scheduler resolves by itself after a while” (Incorrect): That's a different problem, and this one doesn't resolve.
  • “A memory leak caused by tasks that use too much stack space” (Incorrect): Memory isn't the cause here.
  • “A watchdog fault, so the watchdog timer should be removed” (Incorrect): The watchdog is what would rescue the device.
  • “A deadlock: take mutexes in the same order in every task” is correct.

    Neither task can continue while each waits for the other.

    • Also: hold mutexes as briefly as possible.

📖 Revise: Lesson 2: RTOS tasks, queues and priorities

Question ID sp-iot-04-q2 · Report a mistake in this question
3. A device's watchdog is fed from a timer interrupt. The main loop hangs for days without a reset. Why?

Why each option is right or wrong

  • “The watchdog timer is too long, so it should be set to one second” (Incorrect): The length doesn't matter if it keeps being fed.
  • “Watchdogs only work on devices that run an RTOS” (Incorrect): Watchdogs are hardware and work without one.
  • “The main loop must call the watchdog's reset function itself” (Incorrect): That's what feeding after progress means; the timer prevents it.
  • “The timer keeps feeding it while the main work is stuck” is correct.

    Feed only after real progress.

    • Then: the watchdog resets a hung device.

📖 Revise: Lesson 3: Watchdogs and recovering by yourself

Question ID sp-iot-04-q3 · Report a mistake in this question
4. JalSetu wakes and finds RTC memory starting JS1, but the CRC doesn't match. What should it do?

Why each option is right or wrong

  • “Use the data anyway, since the magic marker shows it's the right format” (Incorrect): The CRC shows the data is damaged.
  • “Stop and wait for someone to visit and reset the device by hand” (Incorrect): The device should recover by itself.
  • “Repeat the read until the CRC matches, however long that takes” (Incorrect): Damaged data won't fix itself.
  • “Start from safe defaults, count a cold boot, and carry on with the wake” is correct.

    Damaged state mustn't crash or mislead the device.

    • Kit: Project 1's test 6.

📖 Revise: Lesson 4: Keeping state safe: RTC memory, flash and power cuts

Question ID sp-iot-04-q4 · Report a mistake in this question
5. What's the safest way to replace a small settings file in flash?

Why each option is right or wrong

  • “Overwrite the file in place, byte by byte, as quickly as possible” (Incorrect): A power cut halfway leaves a broken file.
  • “Delete the old file first, then write the new one in its place” (Incorrect): A cut between the two leaves no settings at all.
  • “Keep the settings only in RAM, so flash isn't involved at all” (Incorrect): They'd be lost on every reset.
  • “Write a new file under a temporary name, then rename it” is correct.

    The old file stays intact until the rename.

    • Kit: buf.ack in Project 2.

📖 Revise: Lesson 4: Keeping state safe: RTC memory, flash and power cuts

Question ID sp-iot-04-q5 · Report a mistake in this question
6. A status message shows 96 sensor errors in a day, and the device is still connecting. What's the most likely problem?

Why each option is right or wrong

  • “The Wi-Fi network has gone down at the site for the whole day” (Incorrect): The device is connecting.
  • “The battery is flat, so the device can't wake up any more” (Incorrect): It's waking and sending status.
  • “The server is rejecting the readings because they're too frequent” (Incorrect): The errors are on the device.
  • “The sensor or its wiring has failed, while power and Wi-Fi still work” is correct.

    Every reading failed, but power and network work.

    • Plan: a visit with a new sensor.

📖 Revise: Lesson 5: Diagnosing faults in the field

Question ID sp-iot-04-q6 · Report a mistake in this question
7. What does "real-time" mean in a real-time operating system?

Why each option is right or wrong

  • “Tasks run as fast as the chip allows, ignoring their priorities” (Incorrect): Priorities are how timing is kept.
  • “The operating system shows the current time on a screen” (Incorrect): It's about timing, not clocks on screens.
  • “Tasks can only run when the device is connected to the internet” (Incorrect): RTOS tasks run without a network.
  • “Tasks run predictably on time, even if not as fast as possible” is correct.

    Timing can be relied on.

    • Example: a sensor task that keeps its schedule.

📖 Revise: Lesson 2: RTOS tasks, queues and priorities

Question ID sp-iot-04-q7 · Report a mistake in this question
8. Why should a device report its reset count and the cause of the last reset?

Why each option is right or wrong

  • “So the device can reset itself more often when it's working well” (Incorrect): Resets aren't a reward.
  • “Because the broker refuses devices that don't report their resets” (Incorrect): Brokers don't require it.
  • “So that the battery lasts longer after each watchdog reset” (Incorrect): Reporting doesn't save battery.
  • “So the team can spot crashes, for example after an update” is correct.

    Rising watchdog resets point to a bug.

    • Then: halt the rollout and investigate.

📖 Revise: Lesson 5: Diagnosing faults in the field

Question ID sp-iot-04-q8 · Report a mistake in this question
9. A mechanical button triggers its interrupt seven times for one press. What's the fix?

Why each option is right or wrong

  • “Use a faster microcontroller so that it counts presses correctly” (Incorrect): Speed makes it count bounces even more reliably.
  • “Press the button more gently so the contacts don't move as much” (Incorrect): Users can't control bounce.
  • “Count every seventh interrupt as one press, as the pattern repeats” (Incorrect): Bounce varies from press to press.
  • “Debounce: ignore any changes for a few milliseconds after the first one” is correct.

    Mechanical contacts bounce.

    • Also: keep the handler short.

📖 Revise: Lesson 1: Interrupts: reacting on time

Question ID sp-iot-04-q9 · Report a mistake in this question
10. A firmware wraps the whole wake in except Exception: pass "to stop crashes". What's the problem?

Why each option is right or wrong

  • “Python doesn't allow except Exception in firmware code” (Incorrect): It's allowed.
  • “It makes the firmware run more slowly on every wake” (Incorrect): Speed isn't the issue.
  • “It stops the watchdog from working when the device is asleep” (Incorrect): The watchdog isn't the problem here.
  • “It hides unexpected errors, leaving the device running in a broken state unseen” is correct.

    Handle expected errors; let others reset the device and be reported.

    • Kit: SensorError is handled, other errors aren't.

📖 Revise: Lesson 3: Watchdogs and recovering by yourself

Question ID sp-iot-04-q10 · Report a mistake in this question

ProLearners Assist