The example starts from a deliberately sloppy setup (eight passages, arbitrary order, temperature 1, answers that sometimes misquote) and adds one control per step.
Outcome of 960 trials per step as controls are added (toy model)
Step
Right (or right refusal)
Refused an answerable question
Wrong, confident
sloppy baseline
40%
19%
41%
+ temperature 0.4
52%
19%
29%
+ only 3 passages
54%
17%
29%
+ best passages at the ends
61%
13%
26%
+ verify and retry
68%
13%
20%
+ refuse if samples disagree
48%
50%
2%
Look at the last row honestly. Hallucination nearly vanishes, but half of all questions now get a refusal, including many that had a right answer. Whether that is acceptable is a product decision, not a technical one. The usual next move is to improve retrieval (module 3), query handling (module 4) and the sources themselves so that fewer questions are refused for lack of evidence.
Measuring it
You cannot reduce what you do not measure. Keep three numbers, never one:
Wrong, confident answers (the hallucination rate): the share of all questions that were answered and wrong.
Wrongly refused: answerable questions that got a refusal.
Unfaithful answers: answers whose claims are not in the cited sources (module 5's check).
Include unanswerable questions in the test set, and change one control at a time so you can attribute the gain. Module 7 covers retrieval metrics and Level 9 covers end-to-end evaluation.
Practical advice
Retrieve fewer, better passages and put the best first.
Set temperature low for factual answers, and use the refusal sentence from module 5.
Verify claims against cited sources and retry or refuse on failure.
Use disagreement as a signal where errors are costly, and accept the extra cost.
Keep the loop short in agents. Check facts before they are written into memory or passed to the next step, so one wrong claim cannot snowball.
Keep sources current. A grounded answer from a stale document is a confident wrong answer (module 9 of this level covers freshness).
Send high-stakes cases to a person, and use their corrections as test cases.
Interview angle
"How would you reduce hallucination in a RAG assistant?" Group the answer by where you act: better context (few, relevant passages, best first), tighter instructions and low temperature, a post-hoc check of each claim against its source, and a decision layer that refuses when evidence is thin or when repeated samples disagree. Then show you understand the trade-off: every control that removes a wrong answer can also remove a right one, so you report wrong, refused and unfaithful rates together, and set the threshold by the cost of each kind of error. Mention that you would test the "lost in the middle" effect on your own model rather than assume it.
Check yourself
In the recipe table the last step cuts wrong answers to 2 percent but correct answers fall from 68 to 48 percent. Is that a good change? (It depends on the cost of a wrong answer. Refusing half the questions is acceptable for a medical or legal assistant and probably not for a casual helper. The table shows the trade, not a winner.)
For the curious What changed in the example code
The prompt builder here was a copy of module 5's; it now has the same fix that stops source text from closing its own fence. The percentages come from a model that samples answers with a chosen randomness and an assumed position penalty. They show how each change moves the numbers in a toy, not how a real model behaves.
What you will use daily at work
Practice
What it looks like
Few, well-ordered passages
Retrieve a wide set, rerank, and send only the best few.
Temperature low for factual answers
Set it in the request, and check that answers repeat.
A "can't find it" path
A refusal rule plus a metric for how often it fires.
Draft: confirm current defaults and limits of your model provider before relying on them.