ML6: Language models and retrieval (RAG)
Search: matching words and matching meaning
🎯You'll be able to
Explain TF-IDF and embedding search, compute a similarity score, and improve word-based search with stemming and a glossary.
🏠The everyday picture
A librarian who only matches the exact words on your slip finds "refund policy" but not "getting my money back". A librarian who knows what you mean finds both. Search systems are one or the other, or a mix.
📘What it really is
-
TF-IDF turns each text into a list of word weights: words frequent in this text but rare across all texts weigh most. Two texts are compared with cosine similarity, from 0 (nothing shared) to 1 (the same words in the same proportions).
-
Strengths of word matching: fast, runs anywhere, exact on names, codes and numbers ("COD", "₹30"). Weakness: it misses different words with the same meaning ("heaviest" and "maximum weight").
-
Helping word matching:
- stemming cuts word endings, so "deliveries", "delivered" and "delivering" match;
-
a glossary adds the documents' words to customers' words ("heaviest" → "maximum weight").
-
Embeddings are lists of numbers from a pretrained model (ML5 Lesson 5) that place texts with similar meanings close together, so "money back" lands near "refund". A vector database stores them and finds the nearest ones quickly.
-
Hybrid search combines both scores and is a common, strong choice.
- The kit uses TF-IDF with stemming and a glossary so it runs offline; an embedding model is a design-notes option (not verified here).
📝A worked example
Input: the kit's 30 answerable test questions (eval/rag_questions.jsonl), scored with "hit at 3": is the right page among the top 3
results? Reference measurements, 2 November 2026:
| Retrieval | Hit at 3 | Answer contains the expected fact |
|---|---|---|
| Plain TF-IDF on sections | 0.9333 | 0.6667 |
| With stemming and a 15-entry glossary | 1.0 | 0.9333 |
The glossary was built by reading these same questions' failures, so these figures are optimistic; Lesson 4 measures it on questions kept
aside. Look at the tokeniser: tokens("Parcels weren't delivered; deliveries failed twice") gives parcel, weren, deliver, delivery, fail,
twice.
🌍Where it's used
- Shop search: word matching for product codes, embeddings for "something like this".
- Legal and policy search: exact phrases matter, so hybrid search is common.
- Support: matching a customer's words to help articles written in the company's words.
📝Case scenario
A bank's help search found nothing for "card stuck in machine" because the page was titled "ATM card retention". Adding synonyms helped a little; adding embedding search found it and many similar cases. They kept word matching too, because customers also typed exact error codes, which embeddings sometimes ranked poorly.
🖱️Try it
Remove the glossary entry for "heaviest" from dakdost/rag.py, rebuild the index and ask "What's the heaviest parcel I can send?". Success:
the answer now quotes the business-account sentence from the pricing page instead of the 30 kg limit, which shows exactly what the
glossary was doing.
⚠️Common mistakes
- Treating similarity as truth. Sign: the top result is shown even when it's unrelated. Fix: a refusal rule (Lesson 3).
- Only embeddings. Sign: exact codes and numbers are missed. Fix: hybrid search.
- Tuning the glossary on your test questions without saying so. Sign: perfect scores. Fix: keep a held-out set (Lesson 4).
🛠️For your project
Project 3's test 2 checks hit at 3 of at least 0.9 on the 30 answerable questions.
✅Quick check
What does stemming do for word matching?
Show the answer
It cuts word endings, so different forms of the same word match.
What can embeddings find that word matching misses?
Show the answer
Different words with the same meaning, such as "money back" and "refund".