Part 10 of 12 · 1 min read
Quick notes
- Behaviour depends on more than code: prompts, model snapshot, settings, tools, index and guardrails all matter.
- Define a release manifest and hash it. The id goes in every trace and run; a diff answers "what changed since yesterday?"
- Pointers, not copies.
prodpoints at an immutable release id; rollback just moves the pointer back. - Pin snapshots, avoid aliases, and treat temperature 0 as a tendency, not a guarantee. Compare via evals, not byte equality.
- Prompts are code: version them, render with strict variables, and pin each in-flight run to its starting release.