Part 1 of 5 · 1 min read
Overview
In this module you will
- Store prompts as versioned files that cannot be silently edited.
- Write regression cases with simple, checkable expectations.
- Compare two prompt versions case by case, and block a change that breaks something that used to work.
- Explain what a prompt test can and cannot prove.
A prompt change feels harmless: reword a sentence, tidy a list. But a prompt is code whose behaviour you cannot read off the text, and small edits can quietly break cases that used to work. Without tests you find out from users. With a case set and a comparison you find out in a minute, before shipping. A good habit is to keep prompts in their own files, separate from application code, so that they can be reviewed, tested and edited by people who do not read the code.
About the examples
The model is the same imitation as in module 1 (a rule program that reads the label line, the JSON instruction and the word limit). That is why a prompt edit can change its results in a predictable way. A real model will show noisier, less predictable differences, so for real models run each case several times, or use enough cases that noise averages out. No real model was called.