CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 4 of 7 · 2 min read

First design

WebSockets, a cache and pub/sub.

Parts.

  • Load balancer: spreads traffic across web servers and WebSocket servers.
  • API servers: stateless HTTP servers for friend lists and profiles. These are common and need no special design.
  • WebSocket servers: stateful servers. Each client keeps one long connection, and updates flow both ways (Level 11, page 2).
  • Location cache: Redis holding the latest location of each active user. Each entry has a time to live (TTL) of about 10 minutes. Every update renews it. When it expires, the user counts as inactive and disappears.
  • User database: profiles and friendships.
  • Location history database: all past points. Not needed for the live feature.
  • Pub/sub (publish and subscribe) server: a light message bus with a channel for each user.

How pub/sub is used. Every user has a channel. When a user's phone sends a location, the WebSocket server publishes it to the user's channel. The handler of each online friend has subscribed to that channel, so it receives the update, computes the distance to its own user, and sends it down the friend's WebSocket if it is within 5 miles.

Flow: a location update.

  1. The phone sends a location over its WebSocket.
  2. The WebSocket server saves it to the history database.
  3. It writes the location to the cache and renews the TTL, and keeps it in a variable of the connection handler.
  4. It publishes the location to the user's channel.
  5. The pub/sub server sends it to every subscriber, which are the handlers of the user's online friends.
  6. Each handler computes the distance between the sender and its own user, using the sender's location from the message and its own user's location from its variable.
  7. If the distance is within 5 miles, it sends the new location and time to its client. Otherwise it drops the message.

Flow: a client starts.

  1. The phone connects and sends its location.
  2. The server saves it in the cache and history, and in the handler.
  3. It loads the user's friend list.
  4. It fetches all friends' locations from the cache in one batch call. A friend with no entry is inactive.
  5. For each location, it computes the distance and sends those within range to the client.
  6. For each friend, it subscribes to the friend's channel, active or not. Inactive channels cost a little memory and no CPU.
  7. It publishes the user's location to the user's own channel.

Pub/sub

Publish and subscribe is a messaging style where senders publish to a named channel and every subscriber of that channel receives the message.

Example: user 1 publishes to channel "user-1", and users 2, 3 and 4, who subscribed, each receive it.

Check yourself: 1. Why does each cache entry have a TTL? 2. Who subscribes to a user's channel? 3. Why subscribe to inactive friends too?