My progress
Menu

Level 7 · Artificial Intelligence and Intelligent Automation · Course 7.3 Automation, Evaluation and Governance

Module 90: AI Application Project

Goal: bring the whole level together into one complete, careful AI application: assess its risks, prepare the approved knowledge, integrate every part, put the safeguards in place, and deploy it for real staff. Then complete the Level 7 final project.

You'll need: everything from Modules 81 to 89: your assistant's retrieval, prompts, tools, approvals, automation, test sets and monitoring, plus your Level 6 deployment set-up.

Lessons

  1. 1Risk15–25 minutes✓
  2. 2Knowledge preparation15–25 minutes✓
  3. 3Integration15–25 minutes✓
  4. 4Safeguards15–25 minutes✓
  5. 5Deployment15–25 minutes✓

🛠️Module practice: The assistant, end to end

  1. assistant-charter.md and an AI risk register with controls and tests.
  2. A prepared, approved, versioned knowledge base with an update and review process.
  3. The fully integrated assistant in your CRM, with tests and a passing eval suite.
  4. A safeguards checklist with evidence, and a red-team record.
  5. A staged, flagged, piloted deployment with a staff guide, assistant-limits.md and a pilot review.

🛠️Level 7 final project: Knowledge-Based AI Assistant

You now have everything you need for the Level 7 project (12 to 20 hours). Your assistant answers from approved material, says when it doesn't know, cites sources, and uses only permitted tools. Show that you have:

  1. Defined who it serves and what it must never do (Lesson 1).
  2. Prepared and chunked the approved knowledge (Lesson 2, Module 86).
  3. Built retrieval and checked it finds the right passages, with hit rate and MRR (Modules 85, 86).
  4. Connected a model API with keys kept secret on the server (Module 84).
  5. Shown sources with every answer (Module 86).
  6. Said "I don't know" when the material doesn't cover the question, tested (Module 86).
  7. Given tools the smallest permissions and required approval for any action (Module 87).
  8. Tested against a planted prompt-injection document and shown the protections hold (Module 89).
  9. Built a small test set and measured correctness (Module 89).
  10. Logged questions and answers without storing unnecessary personal data (Module 89).
  11. Deployed it behind login (Lesson 5).
  12. Documented its limits and risks (Lesson 5).

How it's assessed: fundamentals 15%, API integration 15%, retrieval and grounding 20%, evaluation and safety 25%, and this final app 25%. To pass you need 70%, with grounded answers, honest uncertainty and restricted tools all demonstrated. An assistant that invents answers, leaks data across users or acts without approval doesn't pass, whatever its score.

✅Module check

Explain in a few sentences, in your own words:

How do you build and launch an AI assistant responsibly?

Show a good answer

A good answer mentions: a written purpose, users, limits, owner and oversight level, with an AI risk register whose controls are tested; an approved, cleaned, classified, versioned knowledge base kept current and controlled; integration following the architecture (permissions at the route, retrieval, tools and approvals; grounded, cited, checked answers; streaming; logging) with tests and evals; a full set of safeguards with evidence and red-teaming; and a gradual deployment through CI/CD, staging, a flagged pilot and a wider release, with staff guidance, documented limits, ongoing reviews and evals before any model change.

📝Module quiz

10 questions · 10 marks each · 100 marks in all · pass mark 70% · retry as often as you like ·

1. Why write down what the assistant must never do?

Why each option is right or wrong

  • “Because the model needs a written list before it can run” (Incorrect): The model runs without one; the list is for your design and tests.
  • “So the list can be pasted into the prompt as the only safeguard” (Incorrect): Prompt rules alone can be bypassed; limits need code and tests too.
  • “So the project documentation looks more complete” (Incorrect): The value is in enforcing limits, not in the paperwork.
  • “It sets limits that design, tools and tests can enforce” is correct.

    A written list, such as "doesn't give legal advice, doesn't reveal other customers' data, doesn't send emails without approval", turns vague worries into concrete rules. Each rule can then be built into the tools, permissions and test cases, and checked.

    • Share it: with everyone working on the assistant.
    • Link each rule: to a safeguard and a test.

📖 Revise: Lesson 1: Risk

Question ID module-90-q1 · Report a mistake in this question
2. Which is an AI-specific risk beyond ordinary security?

Why each option is right or wrong

  • “A link on the help page that leads to a missing page” (Incorrect): Broken links affect any website, not specifically AI.
  • “A database that responds slowly at busy times” (Incorrect): Slow databases are an ordinary performance issue.
  • “A domain name that expires because nobody renewed it” (Incorrect): Domain renewal is an ordinary operational risk.
  • “Acting on a hallucinated answer” is correct.

    Ordinary apps fail in familiar ways, like crashes or broken links. AI adds new ones: a confident but invented answer that a person or system acts on, such as quoting a refund policy that doesn't exist. That needs grounding, citations and human checks.

    • Others: prompt injection, data leaking through answers.
    • Plan: for each one in your risk list.

📖 Revise: Lesson 1: Risk

Question ID module-90-q2 · Report a mistake in this question
3. Why are most assistant problems knowledge problems?

Why each option is right or wrong

  • “Retrieval systems rarely find the right passage” (Incorrect): Retrieval usually works well when the right documents exist.
  • “Language models get most factual questions wrong” (Incorrect): Models do well with good sources; the gap is often the sources.
  • “Users tend to ask unclear questions on purpose” (Incorrect): Unclear questions happen, but missing documents are a bigger cause.
  • “Missing or outdated documents give wrong answers anyway” is correct.

    Even a good model and good retrieval can only work with the documents you provide. If the right policy isn't there, or an old version is, the answer will be wrong or missing. Fixing the knowledge base often helps more than changing the model.

    • Check: "couldn't find" logs for gaps.
    • Assign: an owner for each document.

📖 Revise: Lesson 2: Knowledge preparation

Question ID module-90-q3 · Report a mistake in this question
4. Why must withdrawn documents be removed from the index?

Why each option is right or wrong

  • “Because the index fills up and stops accepting chunks” (Incorrect): Space isn't the main issue; outdated answers are.
  • “Because index entries expire after a few months anyway” (Incorrect): Index entries usually stay until deleted.
  • “To save money on the storage used by the index” (Incorrect): Savings are small; the real reason is accuracy.
  • “Otherwise the assistant can still repeat them” is correct.

    Deleting a document from the shared drive doesn't remove its chunks from the search index. Until they're deleted there, the assistant can retrieve and quote an outdated or withdrawn policy as if it were current.

    • Process: withdrawal triggers index removal.
    • Check: search for the old text after removal.

📖 Revise: Lesson 2: Knowledge preparation

Question ID module-90-q4 · Report a mistake in this question
5. Where in the flow are permissions enforced?

Why each option is right or wrong

  • “Only in the prompt, by telling the model what's allowed” (Incorrect): Prompts can be ignored or manipulated; checks belong in code.
  • “Only at login, since signed-in users have been checked” (Incorrect): Being signed in doesn't mean being allowed everything.
  • “Only in retrieval, since that's where the data comes from” (Incorrect): Tools and approvals can also act or reveal data; check them too.
  • “At the route, in retrieval, in every tool and at approval” is correct.

    Each step is a chance to leak data or act wrongly. The route checks who is signed in, retrieval filters by what they may see, each tool checks their rights again, and approvals check the approver. If one layer fails, the others still protect.

    • Not in the prompt: prompts can be bypassed.
    • Test: each layer separately.

📖 Revise: Lesson 3: Integration

Question ID module-90-q5 · Report a mistake in this question
6. Why show sources and an accuracy note on every answer?

Why each option is right or wrong

  • “To increase the number of tokens and improve answers” (Incorrect): More tokens don't improve answers; sources help users verify.
  • “So the assistant looks more academic and formal” (Incorrect): Appearance isn't the reason; checking and trust are.
  • “Because regulations require a note on all AI answers” (Incorrect): Rules vary; the main reason is helping users check.
  • “So users can check and don't over-trust” is correct.

    Sources let users open the original document and confirm the answer. A short note, such as "AI answers can be wrong; check important details", reminds them to verify before acting on something important.

    • Make sources clickable: to the exact section.
    • Keep the note short: so people read it.

📖 Revise: Lesson 4: Safeguards

Question ID module-90-q6 · Report a mistake in this question
7. What's red-teaming?

Why each option is right or wrong

  • “Hiring outsiders to steal data from competitors' systems” (Incorrect): Red-teaming tests your own system, with permission.
  • “Letting real users test new features before launch” (Incorrect): That's a pilot or beta test; red-teaming deliberately tries to break things.
  • “Reviewing the code for style and naming problems” (Incorrect): Code review is useful, but red-teaming attacks the running system.
  • “Deliberately trying to make your own system misbehave” is correct.

    Red-teaming means testing your assistant like an attacker or a careless user would: trying prompt injection, asking for other people's data, pushing for harmful advice. Every failure you find becomes a fix and a test, before real users find it.

    • Include: people outside the build team.
    • Repeat: after major changes.

📖 Revise: Lesson 4: Safeguards

Question ID module-90-q7 · Report a mistake in this question
8. Why link every safeguard to a test?

Why each option is right or wrong

  • “To slow down releases so changes are made carefully” (Incorrect): Tests protect quality; slowing releases isn't the aim.
  • “Because a test is needed before a safeguard can be written” (Incorrect): You can write either first; the link proves the safeguard works.
  • “To satisfy the model, which checks the tests before answering” (Incorrect): The model doesn't read your tests; they're for your team.
  • “To show it works and catch breakage” is correct.

    A safeguard you haven't tested might not work, and a later change can quietly break one that did. A linked test proves it works now, and fails loudly if it stops working.

    • Example: a test that user A can't retrieve user B's documents.
    • Run: in CI on every change.

📖 Revise: Lesson 5: Deployment

Question ID module-90-q8 · Report a mistake in this question
9. Why start with an internal pilot?

Why each option is right or wrong

  • “To hide the feature from customers permanently” (Incorrect): A pilot is a first step towards launch, not a way to hide it.
  • “To skip formal testing, since staff will test it” (Incorrect): A pilot adds to testing; it doesn't replace it.
  • “Because pilots don't cost anything to run” (Incorrect): Pilots take time and effort; they're worth it for safety.
  • “To find problems with a small, trusted group first” is correct.

    Staff who know the subject can spot wrong answers quickly and report them, without customers being affected. The pilot shows real usage, gaps in documents and confusing behaviour, which you fix before a wider launch.

    • Collect: feedback and logs from the pilot.
    • Set: clear criteria for moving on.

📖 Revise: Lesson 5: Deployment

Question ID module-90-q9 · Report a mistake in this question
10. What should you do before switching to a new model version?

Why each option is right or wrong

  • “Switch immediately, since newer versions are better” (Incorrect): Newer can be worse for your specific tasks.
  • “Delete your old tests, since they were for the old model” (Incorrect): Old tests are exactly what tells you whether the new model works.
  • “Ask users to test it in production and report problems” (Incorrect): Users shouldn't be the first to find problems.
  • “Run your evals in staging and compare” is correct.

    A new model version can be better overall but worse on your particular tasks, or change its output format. Running your evaluation set in staging and comparing scores, costs and safety results shows whether to switch.

    • Check: safety cases still pass 100%.
    • Keep: the old version ready to roll back.

📖 Revise: Lesson 5: Deployment

Question ID module-90-q10 · Report a mistake in this question

ProLearners Assist