ML6: Language models and retrieval (RAG)
Prompts, structured output and tool calls
🎯You'll be able to
Write a prompt that keeps an LLM to the retrieved text, ask for structured output and validate it, and explain how tool calling works.
🏠The everyday picture
A new assistant given a clear brief ("answer only from these pages; if they don't cover it, say so; always say which page") does much better than one told "help the customer". A clear brief, and checking their work, is most of the job.
📘What it really is
-
A RAG prompt has: instructions (answer only from the sources, cite them, say when they don't cover the question, keep it short), the retrieved sections with their ids, and the question. Put the instructions in the system message and mark the sources clearly.
-
Structured output: ask for JSON with fixed fields, for example
{"answer": "...", "sources": ["help/refunds.md#..."], "covered": true}. Many APIs can enforce a schema. Then validate it in code: is it valid JSON, are the sources among those retrieved, is the answer short enough? -
Tool calling: you describe functions (name, purpose, parameters) to the model; it can reply with a request to call one, such as
track_parcel(tracking_number). Your code checks the request, runs the function and returns the result. The model never runs code itself. -
Safety: treat the model's output as untrusted input. Check tool arguments, limit what tools can do, and ask a person before anything that can't be undone (ML7 Lesson 4).
-
Evaluation still applies: prompts are code. Change one and re-run the test set (Lesson 4).
📝A worked example
Input: the question "How much is the COD fee?" and the kit's top section, help/cash-on-delivery.md#How the money reaches the sender.
A prompt (a design, not verified against a hosted model in this kit):
System: You answer DakDost customers using only the SOURCES below. Quote the facts you use and list the source ids.
If the sources don't answer the question, set "covered" to false and say you couldn't find it.
Reply as JSON: {"answer": string, "sources": [string], "covered": boolean}.
SOURCES:
[help/cash-on-delivery.md#How the money reaches the sender] The courier collects cash or a UPI payment ... minus a COD fee of 2% (minimum ₹30).
User: How much is the COD fee?
Expected behaviour to check: covered is true, the answer gives "2% (minimum ₹30)", and sources lists only the id above. Your code
rejects any reply whose sources weren't in the prompt. The kit's extractive answer for this question, measured, quotes the same section.
🌍Where it's used
- Support drafting: JSON answers with sources, checked before an agent sees them.
- Order tracking: a tool call to look up a parcel, instead of the model guessing.
- Data extraction: turning emails into structured records, validated against a schema.
📝Case scenario
A team's assistant could call a issue_refund tool. A customer's message persuaded it to call the tool for an amount far above the order.
The team limited the tool to the order's own amount, required a staff click to approve any refund, and logged every tool call for review.
🖱️Try it
Write the prompt above for the question "Do you deliver to Nepal?" with the top three retrieved sections. Success: your instructions make
it clear that the right reply has covered false, and you write the code check that would catch a reply that cites a source not in the
prompt.
⚠️Common mistakes
- Vague instructions. Sign: answers drift beyond the sources. Fix: "only from the sources", with a defined "not covered" reply.
- Trusting the JSON. Sign: the app crashes or shows invented sources. Fix: validate fields and source ids in code.
- Powerful tools with no limits. Sign: the model can do anything a tool can. Fix: narrow tools, check arguments, approve risky actions.
🛠️For your project
The capstone design notes plan the step from the kit's extractive answer to an LLM-written one, with the prompt, the checks and the cost.
✅Quick check
Why validate an LLM's JSON output in code?
Show the answer
It can be malformed or cite sources that weren't given, so code has to check it.
Who runs the function in tool calling?
Show the answer
Your code does, after checking the model's request.