CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 6 of 7 · 2 min read

Trade-offs

What to say out loud.

Trade-offs and what to say

Choice Gain Cost
Redis cache with TTL Fast, inactive users clean up themselves Data lost on failure, rebuilt as updates arrive
Pub/sub per user Simple routing 140 servers, many subscriptions
Subscribe to all friends No logic on friend activity More memory for idle channels
Eventual consistency Cheap and fast A friend's dot may lag a few seconds
Over-provisioned pub/sub Rare resizing Idle capacity
Drop messages with no subscriber Almost no load Missed updates during resubscription

Extension: "nearby random people". Add a way to find people who are not friends. Here a spatial search is needed again. A geohash (Level 11, page 6) can give a channel for each grid cell. Users in a cell subscribe to the cell's channel and publish to it, and the receivers filter by distance. This has more fan-out and more privacy risk.

Interview angle

Answer in your head first, then open it.

Why is this different from nearby places?

Locations move, so writes dominate and the task is delivery, not search.

What is the bottleneck?

Pushing about 14 million messages a second, which needs around 140 pub/sub servers.

How do you route updates?

A channel for each user, with friends subscribed, and a hash ring of pub/sub servers.

What if the cache fails?

Replace it with an empty one. It refills within a few update cycles.

How do you handle privacy?

Opt-in, only friends see locations, and short retention. The case set this aside, but a real design needs it.

One follow-up to try. "A user has 5,000 friends." A good reply: fan-out grows from 40 to 500 for that user's updates, since 10 per cent are online and nearby. Cap the number of friends tracked, or send updates for such users less often, or move very large accounts to a pull model where friends ask for the location on opening the screen.

Check yourself: 1. What is a good reply to a user with 5,000 friends? 2. How would you add nearby random people? 3. What does an empty cache cost after a failure?

Words you will meet

Word Meaning
Fan-out The number of receivers for one message
Pub/sub Publishers send to channels and subscribers receive
Channel A named stream in a pub/sub server
TTL Time after which a cache entry expires
Hash ring A circle of hash values that maps keys to servers
Service discovery A small store that tells programs where services are
Draining Letting a server finish its connections before removal