Module 86: RAG

Why each option is right or wrong

• “Retrain the model on your documents, then answer questions” (Incorrect): RAG doesn't retrain the model; it supplies passages in the prompt at question time.
• “Read the whole library, summarise it, then reply” (Incorrect): RAG searches for the few passages that matter instead of reading everything each time.
• “Receive the question, ask the model, then add sources after” (Incorrect): Sources are found before the answer is written, so the model can use them.
• “Retrieve passages, augment the prompt, generate an answer” is correct.

  Retrieval-augmented generation first searches your own documents for passages related to the question (retrieve), then adds those passages to the prompt (augment), and finally asks the model to write an answer based on them (generate). The model answers from your material instead of only its training.

  • Retrieve: usually embeddings plus keyword search.
  • Generate: with instructions to cite and to admit gaps.

📖 Revise: Lesson 1: RAG architecture

Why each option is right or wrong

• “RAG retrains the model automatically whenever documents change” (Incorrect): RAG doesn't retrain anything; it searches the updated documents.
• “Models can't be retrained once they've been released” (Incorrect): Models can be fine-tuned; it's just slower and costlier than re-indexing.
• “RAG stores the newest facts inside the model's weights” (Incorrect): The facts stay in your documents and index, outside the model.
• “You change and re-index documents; the model stays the same” is correct.

  When a policy changes, you edit the document and re-index its chunks, which takes minutes. Retraining or fine-tuning a model to learn new facts is slow, costly and less reliable, and you'd have to repeat it for every change.

  • Also: you can remove a document and the knowledge goes with it.
  • Fine-tuning: better for style and format than for facts.

📖 Revise: Lesson 1: RAG architecture

Why each option is right or wrong

• “Because vector databases reject chunks without a title” (Incorrect): Vector stores accept plain text; the title helps meaning, not storage.
• “To make every chunk longer so it scores higher” (Incorrect): Length doesn't raise scores; context makes matches more accurate.
• “So chunks can be sorted into alphabetical order in the index” (Incorrect): Order in the index doesn't matter for search; context in each chunk does.
• “So each chunk makes sense alone and matches better” is correct.

  A chunk cut from the middle of a document may say "This applies within 30 days" with no clue what "this" is. Adding "Refund policy > Returns" gives the chunk its context, so both the search and the model understand it.

  • Example: "Refund policy > Returns: Items can be returned within 30 days."
  • Also useful: for showing citations to users.

📖 Revise: Lesson 2: Chunking

Why each option is right or wrong

• “So the search runs faster by comparing fewer chunks” (Incorrect): Overlap creates slightly more chunks; it doesn't speed things up.
• “So duplicate documents can be detected and removed” (Incorrect): Overlap is about boundaries, not finding duplicates.
• “So each chunk is stored twice as a backup” (Incorrect): Only a small slice is repeated, for context; it isn't a backup.
• “So ideas across a boundary aren't cut in half” is correct.

  Fixed-size chunking splits text every few hundred tokens, which can land mid-sentence or mid-idea. Repeating a little text at the start of each chunk, say 10–20%, means an idea split at a boundary appears whole in at least one chunk.

  • Better still: split on headings and paragraphs where possible.
  • Cost: slightly more storage, which is usually worth it.

📖 Revise: Lesson 2: Chunking

Why each option is right or wrong

• “To remove the chunks the user isn't allowed to see” (Incorrect): Permission filtering belongs in the retrieval query, before ranking.
• “To shorten the passages so the answers are briefer” (Incorrect): Reranking reorders passages; it doesn't shorten them.
• “To sort the passages by date, newest first” (Incorrect): A reranker sorts by relevance to the question, not by date.
• “A reranker judges relevance more carefully” is correct.

  The first search is fast but rough: it compares vectors and may rank a loosely related chunk above the one that answers the question. A reranker reads the question and each candidate together and scores them more precisely, so the best few go to the model.

  • Typical flow: fetch the top 20–50, rerank, keep the top 5.
  • Trade-off: a little extra time and cost.

📖 Revise: Lesson 3: Retrieval and reranking

Why each option is right or wrong

• “After the answer is shown, by hiding restricted parts” (Incorrect): By then the restricted content has been sent and possibly read.
• “In the model's prompt, by telling it not to reveal restricted text” (Incorrect): A prompt can be ignored or bypassed; filtering must happen in code.
• “Only at sign-in, since logged-in users are trusted” (Incorrect): Logged-in users still have different rights; check on every search.
• “In the retrieval query itself” is correct.

  If the search returns chunks the user isn't allowed to see, they can leak into the answer however the prompt is worded. Filtering in the query, for example WHERE department = user's department, means restricted chunks aren't retrieved for that user at all.

  • Store with each chunk: who may see it.
  • Test: that one user can't get another's documents.

📖 Revise: Lesson 3: Retrieval and reranking

Why each option is right or wrong

• “Mixing the passages with the model's general knowledge freely” (Incorrect): That's what grounding tries to prevent; general knowledge may not fit your case.
• “Running the model on your own local servers” (Incorrect): Where the model runs is separate from grounding.
• “Connecting the model to live internet search results” (Incorrect): Web search can feed grounding, but grounding means answering from supplied sources.
• “Answering only from the supplied passages” is correct.

  A grounded answer uses the retrieved passages as its source of facts, rather than the model's general knowledge, which may be out of date or wrong for your organisation. If the passages don't cover the question, a grounded assistant says so.

  • Prompt: "Answer only from the passages below, and cite them."
  • Check: with faithfulness tests.

📖 Revise: Lesson 4: Grounding and citations

Why each option is right or wrong

• “It hides errors in the system from the user” (Incorrect): It's the opposite: it's an honest report that the information isn't there.
• “It saves tokens, so it keeps costs low” (Incorrect): Saving tokens is a side effect; honesty is the point.
• “Users prefer short answers to detailed ones” (Incorrect): Users want correct answers; this is better than a wrong one.
• “It's honest and shows which documents are missing” is correct.

  An invented answer can mislead someone into a wrong decision. Saying the information wasn't found is honest, and logging these replies shows you which topics your documents don't cover yet, so you can add them.

  • Track: the "couldn't find" rate over time.
  • Offer: a next step, like contacting the team.

📖 Revise: Lesson 4: Grounding and citations

Why each option is right or wrong

• “Faithfulness is about users trusting the app; correctness is about facts” (Incorrect): Faithfulness measures whether the answer sticks to its sources.
• “Correctness is about speed; faithfulness is about accuracy” (Incorrect): Neither measures speed.
• “They're two names for the same measurement” (Incorrect): They differ: an answer can follow outdated passages faithfully and still be wrong.
• “Faithfulness: sticks to passages; correctness: actually right” is correct.

  A faithful answer says only what the passages support. A correct answer is true. They can differ: if a passage is outdated, a faithful answer repeats the outdated fact. Measuring both shows whether a problem is in the model or the documents.

  • Unfaithful: the model added or changed facts.
  • Faithful but wrong: fix the documents.

📖 Revise: Lesson 5: Evaluating RAG

Why each option is right or wrong

• “It isn't needed, since a model is more consistent than people” (Incorrect): Consistent isn't the same as right; it can be consistently wrong.
• “Because LLM judges can't read the answers they grade” (Incorrect): They read them; the concern is whether they grade accurately.
• “Because human judgements are faster to collect” (Incorrect): Humans are slower; that's why judges are used, and checked.
• “The judge can be wrong or biased too” is correct.

  Using a model to grade answers saves time, but it can misjudge, favour longer answers, or miss subtle errors. Comparing its scores with careful human ratings on a sample shows whether you can trust it.

  • Re-check: when you change the judge model or prompt.
  • Keep: some human review for high-risk cases.

📖 Revise: Lesson 5: Evaluating RAG