CSE Roadmap

Common tier · about 3 hours · 12 parts

Part 4 of 12 · 2 min read

Retrieval: filter before search, not after

Filtering results after a global search both leaks and loses them.

Think of searching a shared filing room for your own folder, versus searching only your own filing cabinet. Searching the whole room and discarding other people's folders afterward still means you walked past them. And if your own folder is thin, you might come away with almost nothing.

In plain words: filtering out other tenants' documents after a global top-k search exposes those documents to the process doing the filtering. It can also starve a small tenant of nearly all its own relevant results.

A worked example. In a 20,000-document index split across 100 tenants of very different sizes, one tenant had only 163 documents. When the search ran globally first and filtered afterward, it got a mean of just 0.07 of its own results in a top-5 search. Almost all five results-worth of attention was spent on other tenants' documents. Most of those were simply dropped, but still pulled into memory first. Searching only within a namespace scoped to that tenant instead returned 5 of 5 every time, regardless of how small the tenant was.

The key lines.

# from code/mini-tenancy/tenancy.py
pool = mine if mode == "namespace" else range(n_docs)
top = sorted(pool, key=score, reverse=True)[:k]
got += sum(1 for i in top if owner[i] == tenant_rank); foreign += sum(1 for i in top if owner[i] != tenant_rank)

Read it as: the namespace version restricts the candidate pool to the tenant's own documents before ranking anything. The post-filter version ranks everyone first and only checks ownership afterward. By the time the post-filter check runs, other tenants' documents have already been scored and held in memory. The leak has already partly happened, even if the result is then discarded.

Namespace, pre-filter, post-filter

A namespace is a separate partition of an index per tenant; a pre-filter restricts candidates before similarity search runs; a post-filter drops results only after the search has already run globally.

Example: the namespace search above, which never considers another tenant's documents at all, against the post-filter search, which considers all 20,000 before discarding most of them.