Part 8 of 13 · 1 min read
How to build it so it holds up
- Log spans with arguments, statuses, durations, token counts and versions from day one.
- Return structured errors from tools and never let a loop treat an error as an observation silently.
- Compare with a healthy run before reading prompts.
- Write rival hypotheses and what each predicts, then choose the cheapest test.
- Change one thing at a time.
- Read the false positives of every alert.
- Use rules for structure and evaluations for quality; a fluent wrong answer needs a grader or a human.
- Put every fixed failure into a regression set.
- Bound loops and retries and alert on the bound being hit.