CSE Roadmap

Core tier · about 4 hours · 12 parts

Part 9 of 12 · 2 min read

Mistakes that leave personal data exposed

Common mistakes, and what the example does not do.

Mistakes that leave personal data exposed

  • Sending the whole customer record to the model.
  • Logging full prompts and tool results forever.
  • Deleting the source record but not its summary or embedding.
  • Trusting a regex redactor (or a model) as complete.
  • Storing the token vault next to the logs.
  • Evaluating detectors on English test data only.
  • No retention limit on traces and eval sets.
  • Ignoring provider-side retention and region.

What the example does not do

About this example What the example does not do
  • Small, hand-written, fictional corpus; the same person wrote rules and labels, so the 100 percent precision is optimistic.
  • No named-entity recognition, non-English text, addresses, dates of birth or free-text identifiers.
  • Vault in memory, with a fixed demo key; no key rotation, access control or audit.
  • Erasure on an in-memory store, not on real databases, backups, caches or provider logs.
  • No re-identification analysis: combinations of harmless fields can identify a person.
  • No legal analysis, and no differential privacy or anonymisation guarantees.
  • Optional real experiment: take 200 real (consented, internal) support tickets, label PII by hand and compare your detector with an NER-based one per type.

Check yourself

You delete a customer raw record. Is their data gone?

Show answer

Not if copies derive from it. With four stored copies, deleting the raw record left 3; following derivation links removed all 4.

For the curious Detection is imperfect, so minimise first

Regex rules found 13 of 17 labelled items (recall 76 percent, precision 100 percent) on a 20-text set written for the example; they missed an obfuscated email, a name without a title and a spelled-out phone number. A Luhn check stopped order numbers and IBAN digit blocks being redacted as cards. The best defence is to not send the field at all: the three example tasks needed 2, 3 and 3 of 12 fields and none of the six personal ones. This is not legal advice.

What you will use daily at work

Practice What it looks like
Pass only the fields the task needs Do not rely on redaction after the fact.
Use stable tokens with a protected vault Detokenise only where a tool truly needs the real value.
Map every copy Prompts, logs, traces, memory, embeddings, eval sets, provider logs.
Erase by lineage and age Follow derived copies; apply retention limits.