CSE Roadmap

Core tier · about 4 hours · 11 parts

Part 6 of 11 · 1 min read

How to build it so it holds up

  • Give every factual claim a source (a passage, a tool result) and show it to the user or the checker.
  • Extract and verify numbers, dates, names and identifiers by code against the source.
  • Use an entailment model or a calibrated LLM judge for paraphrase, and measure its error rates (module 4).
  • Allow and reward abstention: "not in the provided documents" is a correct answer.
  • Use self-consistency selectively (high-stakes answers, no context available), with k of 3 to 5, and treat low agreement as a trigger for abstention or review.
  • Measure on held-out claims from real traces (module 2), sliced by kind of claim.
  • Log unsupported claims in production and sample them for review (module 8).