Part 6 of 7 · 4 min read
Exercises
Try each exercise before opening its hint.
Easy
Easy 1: Settings that fail fast (25 minutes)
In code/config-and-logs, run pip install pydantic-settings, copy .env.example to .env, fill in a fake key, and print load_settings(). Check that the key is hidden. Remove the key and run again; read the error. Then confirm git status does not list .env (copy the .gitignore into a new repo to test).
Done when: you have seen both the hidden value and the start-up failure.
Easy 2: Sort the log lines (15 minutes)
For each, say "safe", "unsafe" or "safe if trimmed": (a) the request ID, (b) the full Authorization header, (c) the model name, (d) the user's full prompt, (e) the token counts, (f) a database URL with a password.
Done when: you can give a one-line reason for each.
Hint
logs are copied to many places; ask who can read them later.
Easy 3: Where does the value come from? (15 minutes)
A setting APP_LLM_NAME is small-1 in the class, from-file in .env, and from-shell in the shell. Predict the value for each combination of the three being present, then check by running the example.
Done when: your predictions and the runs agree for all combinations you tried.
Hint
the order is shell variable, then .env file, then the default. Try removing one source at a time.
Medium
Medium 1: Add a setting (25 minutes)
Add max_steps: int = 10 to Settings, with a rule that it must be between 1 and 50 (use Field(ge=1, le=50)). Add three tests: the default, a valid override, and an invalid value that raises ValidationError. Update .env.example.
Done when: pytest passes and the example file lists the new name.
Hint
from pydantic import Field, then max_steps: int = Field(default=10, ge=1, le=50).
Medium 2: Extend the scanner (30 minutes)
Add a pattern for the key format of a provider you use (check its docs, module 8). Add a test with a fake example and a test for a false positive you care about (for example the text sk-learn in a sentence). Run the scanner over the project folders from modules 1 to 5.
Done when: the scanner finds nothing in your own projects and flags a deliberately planted fake key.
Hint
check provider docs for the key prefix and length; make the pattern as specific as you can.
Medium 3: Redact in the formatter (30 minutes)
Make JsonFormatter replace any value whose key name contains key, token, secret or password with "***", including inside nested dictionaries. Test with a log call that passes a nested dictionary.
Done when: the nested secret is masked in the output and other fields are untouched.
Hint
write a small recursive function over dict and list values.
Hard
Hard 1: Add a request ID to every line (40 minutes)
Using contextvars.ContextVar, store a request ID once at the start of a request and make JsonFormatter include it on every log line automatically, without passing it to each call. Test that two interleaved requests do not mix their IDs.
Done when: a log line written deep inside a function shows the right request ID.
Hint
set the variable in a middleware or at the top of your handler; read it with request_id.get("-") in the formatter.
Hard 2: A cost-aware log (50 minutes)
Write a function log_model_call(log, model, input_tokens, output_tokens, latency_ms) that logs those fields plus an estimated cost, computed from a price table in settings (price per million tokens, input and output). Write a test that runs five calls and sums the logged costs from the captured output.
Done when: the total from the logs equals the total you calculate by hand.
Hint
keep money as an integer number of micro-dollars to avoid floating-point rounding surprises in totals.
Hard 3: A pre-commit guard (50 minutes)
Write a pre-commit hook script (any language) that runs the secret scanner on staged files only and refuses the commit with a clear message if it finds something. Install it in a practice repo and try to commit a file with a fake key, then a clean file. Describe one way the hook can be bypassed and what catches it later.
Done when: the dirty commit is refused, the clean commit passes, and you named the bypass and the second line of defence (CI).
Hint
git diff --cached --name-only lists staged files; the hook is .git/hooks/pre-commit and must be executable.