CSE Roadmap

Core tier · about 4 hours · 12 parts

Part 5 of 12 · 2 min read

Deleting the record is not deleting the person's data

Everything derived from a record is still that person's data, even after the record itself is gone.

Think of shredding a single photo negative while four prints made from it sit untouched in a drawer. The negative is gone. But the image is still very much in circulation.

lineage is a record of which items were derived from which, retention is how long each copy is kept, and erasure is deleting a person's data on request. Following lineage when erasing means every summary, embedding or log line derived from a deleted record gets deleted too, while deleting only the raw record leaves every one of those derived copies standing, still carrying the same personal data under a different name.

A worked example. With four stored items for one customer -- a raw record, a summary made from it, an embedding made from that summary, and a log line -- erasing only the raw record leaves 3 of the 4 copies behind, because nothing told the store those three derived items belonged to a now-deleted person. Erasing by lineage instead finds every item whose derivation chain traces back to the raw record (including the embedding, two steps removed) and removes all 4, while correctly leaving a different customer's unrelated record untouched.

The key lines. From code/mini-privacy/privacy.py:

# from code/mini-privacy/privacy.py
def erase_lineage(self, subject):
    doomed = {i for i, v in self.items.items() if v["subject"] == subject}
    changed = True
    while changed:
        changed = False
        for i, v in self.items.items():
            if i not in doomed and any(s in doomed for s in v["derived_from"]):
                doomed.add(i); changed = True
    for i in doomed: del self.items[i]
    return len(doomed)

Read it as: the loop keeps re-scanning for new items whose derived_from points at anything already marked doomed. It only stops once a full pass adds nothing new. This is what lets the embedding get caught even though it derives from the summary, not from the raw record directly. The "doomed" set grows one derivation hop at a time until the whole chain is covered.

Retention, erasure, lineage

Retention is how long each copy is kept. Erasure is deleting a person's data on request. Lineage is records of which items were derived from which, so derived copies can be found.

Example: "3 copies left after a naive delete, 0 after following lineage" is the entire argument for recording lineage in the first place -- without it, erasure is a promise the system has no way to actually keep for anything beyond the one record someone remembered to delete by hand.