Part 9 of 11 · 1 min read
Quick notes
- Define it before you detect it. For agents the useful split is unfaithful to the context (contradicts or goes beyond it) and unsupported by the world (invented, with no context to check).
- Two detectors: a faithfulness check that tests each claim against the trusted context, and a self-consistency check that samples the same question several times.
- A cheap lexical check has predictable blind spots: it passes changed numbers and flipped meaning, and fixing those with number and negation checks raises false alarms on paraphrases. The rules need a held-out set and an entailment model or calibrated judge.
- Self-consistency flags guesses, not confident errors: a model that gives the same wrong answer every time passes it.
- Neither detector replaces grounding: retrieve and cite, constrain answers to the context, let the agent abstain, check numbers against tools, and keep a human gate for high-stakes claims.