Part 4 of 5 · 3 min read
Exercises
Easy
Easy 1: Name the parts (15 minutes)
Take any prompt you have used with a chatbot and mark which sentences are role, task, context, constraints, format and input. Note which parts are missing.
Done when: you have a labelled copy and a list of missing parts.
Easy 2: Fix the vague rules (15 minutes)
Rewrite these as checkable rules: "Be concise", "Give a few examples", "Respond soon", "Use an appropriate tone". Say how a test could check each rewrite.
Done when: each has a number or a concrete test.
Hint
"at most 3 sentences", "exactly 3 examples", "reply within 2 seconds", "no slang or contractions".
Easy 3: Predict the lint (20 minutes)
For three small PromptSpec objects you invent, write down what lint will return before running it, then run it.
Done when: you have three predictions and three results, with any differences explained.
Medium
Medium 1: A new task (60 minutes)
Build a PromptSpec that asks the imitation model for a one-word answer (the model recognises the phrase "one word") for the same tickets. Evaluate it with a small variation of evaluate that accepts plain labels.
Done when: you report accuracy and explain what you changed in evaluate.
Hint
the imitation answers with just the label when the prompt contains "one word".
Medium 2: Add a linter rule (60 minutes)
Add a check that fails when the output format asks for JSON but neither the format nor the constraints mention the field names that the code will read. Test with a passing and a failing spec.
Done when: two tests pass and the rule has a clear message.
Hint
a regular expression on the output format text for "label" and "reason" is enough.
Medium 3: Ablate the constraints (60 minutes)
Split the two constraints into separate ablations (remove only the word limit, then only "pick exactly one"). Report the effect of each.
Done when: a table has both rows and one sentence on what you learned.
Hard
Hard 1: Design a prompt for a real model (120 minutes)
Using a real hosted or local model you can access, write a spec for the ticket task, run it on 20 tickets you write yourself, and run four ablations. (I did not run this one, so your numbers are the ground truth.)
Done when: you have a table of results, and you state which parts mattered for the real model and where it differed from the imitation.
Hard 2: Prompt size budget (120 minutes)
Make a prompt with a long context (a 2,000-word policy). Write a function that trims the policy to a token budget by keeping only paragraphs that share words with the ticket. Show a case where trimming helps and a case where it removes something needed.
Done when: both cases are tests, and you describe the risk.
Hint
split on blank lines, lower-case, and compare word sets.
Hard 3: A prompt registry (120 minutes)
Build a small registry that stores specs by name and version in JSON files, loads them, renders them, and refuses to load a spec that fails lint. Add a command that prints the difference between two versions.
Done when: a test loads two versions and shows the diff (this prepares module 5).
Hint
dataclasses.asdict and json make the round trip easy. Compare field by field for the diff.