Part 4 of 11 · 2 min read
A model that knows a fact repeats it; a model that's guessing usually doesn't
Ask the same question five times: disagreement across the answers is itself a signal, with no source text required.
Think of asking a friend the same trivia question five different times, spaced out, without telling them you already asked. If they really know the answer, they will give the same one each time. If they are guessing, chance will usually produce at least one different answer somewhere in the five.
In plain words: a self-consistency check samples several answers to the same question and measures how much they agree, abstention is the model declining to answer because it lacks support, and consistent hallucination is the same wrong answer coming back on every single sample -- and self-consistency catches the second case by its disagreement but is blind to the third, because a consistently wrong model looks exactly like a model that knows the fact.
A worked example. In a simulation where a model knows 70% of facts (always answering "A" for those) and guesses uniformly among 6 wrong answers for the rest, sampling k=5 times flags 99.7% of genuine unknowns as low-agreement, with zero false alarms on known facts. But when 60% of the unknown facts get one fixed wrong answer every time instead of a random guess, that same k=5 check's recall collapses to 39% -- it cannot tell a confidently, consistently wrong answer from a genuinely known one, because both look like perfect agreement.
The key lines. From code/mini-faithful/faithful.py:
# from code/mini-faithful/faithful.py
def sample_answers(knows, k, rng, n_wrong=6, confident_wrong=0.0):
if knows:
return ["A"] * k
if rng.random() < confident_wrong:
return ["W0"] * k
return [f"W{rng.randrange(n_wrong)}" for _ in range(k)]
Read it as: the function has exactly three branches. The middle one, confident_wrong, is the model simulated as consistently wrong. It returns the same fixed wrong answer k times. That looks identical to the "knows" branch to any agreement-based detector, even though the content is false.
Self-consistency, abstention, consistent hallucination
A self-consistency check samples several answers and measures their agreement. Abstention is the model declining to answer because it lacks support. Consistent hallucination is the same wrong answer on every sample.
Example: with 60% of unknown facts answered the same wrong way every time, a 5-sample self-consistency check's recall fell from 99.7% (random guessing) to 39% -- consistent hallucination is the blind spot a faithfulness check (parts 1 and 2) exists to cover when there's no context to check against.