Part 8 of 12 · 2 min read
Answering data-privacy questions in an interview
How do you handle PII in an LLM application?
"I list every place data is copied, then minimise what reaches the model, tokenise what must reach it with the mapping in a separate vault, redact traces and logs, set retention per data class and make erasure follow lineage to summaries and embeddings. I measure the example's detectors: in a small test regex rules had 100 percent precision and 76 percent recall, so I do not treat redaction as a guarantee." Follow-up: "What would you use instead of regex?" A named-entity model or a library such as Presidio, evaluated on my own data, plus the structural controls.
A user asks you to delete their data. What do you delete?
"The raw records, and everything derived from them: summaries, embeddings, memory entries, cached results, log lines and eval examples, using lineage records. In the example's test deleting only the raw record left three copies. Then I check backups and provider-side retention and record that it was done." Follow-up: "What about a model fine-tuned on the data?" That is much harder; avoid training on personal data without a plan, or keep it out of training sets in the first place.
Redaction or pseudonymisation?
"Redaction is irreversible and simple but loses the link between mentions. Pseudonymisation with stable tokens lets the model reason about the same entity and lets a tool get the real value at the boundary, but you now hold a vault that must be secured and is itself personal data. I choose by whether the task needs the identity." Follow-up: "Where does the vault live?" Outside the prompt, logs and traces, with its own access control and retention.
Why do logs and traces matter?
"They copy everything, last longest and are readable by many engineers. The OpenTelemetry content attributes are opt-in because they can hold PII, so I capture content only on purpose, redact before export, and keep retention short." Follow-up: "How do you debug without content?" Log structured fields and token ids, and capture full content for a small consented sample.