Part 6 of 8 · 2 min read
Query service
Cache, browser and filters.
Query path. A request goes through a load balancer to an API server. The server reads the prefix's entry from the trie cache and builds the answer. If the entry is missing, perhaps because a cache server restarted or ran short of memory, the server loads it and puts it back in the cache, so later requests are fast.
Three more speed-ups from the book:
- AJAX requests. The browser fetches suggestions without reloading the page.
- Browser caching. Suggestions change little over an hour, so the browser can reuse them. The book shows Google caching results in the browser for 3,600 seconds, or one hour.
- Sampling on the logging side, as above.
Updating the trie. Two ways. (1) Replace the whole trie each week by swapping in the newly built one. (2) Change a single node directly. The second is slow, because every ancestor holding that query in its top list must change too. A worked example (made up): raising "beer" from 10 to 30 means updating the node for "beer" and the nodes for "b", "be" and "bee", wherever their lists hold it. Direct updates are fine only for a small trie.
Deleting bad suggestions. Hateful, violent or unsafe suggestions must go. A filter layer sits between the trie cache and the API server, and removes unwanted entries from each answer. The filter takes effect at once. The bad queries are also removed from the stored data, so that the next weekly trie is clean.
Read it as
Think of the weekly trie as a printed directory. Direct updates are pencil corrections in the margin, and the filter is a black marker over lines that must not be read.
Check yourself: 1. What happens on a cache miss? 2. Why is changing one node slow? 3. How does the filter layer differ from deleting from the data?