CSE Roadmap

Core tier · about 4 hours · 8 parts

Part 3 of 8 · 2 min read

The two flows

Publishing and building.

Feed publishing. A user sends a post. The book gives the API as POST /v1/me/feed with the post text and an auth token. The path through the system:

  1. A load balancer sends the request to a web server.
  2. The web server checks the token and applies a limit on posts per user per period. This stops spam.
  3. The post service saves the post in the database and the cache.
  4. The fanout service delivers the new post to friends' feeds.
  5. The notification service tells friends there is something new.

Feed building. The reader asks GET /v1/me/feed. A load balancer sends it to a web server, which calls the news feed service. That service reads a list of post ids from the news feed cache, then builds the full feed.

publish:  user -> LB -> web server -> post service -> db + post cache
                                   -> fanout service -> feed cache of each friend
                                   -> notification service

read:     user -> LB -> web server -> feed service -> feed cache (ids)
                                   -> post cache + user cache (full objects)
                                   -> CDN (images, video)

Why two stages on the read side? The feed cache holds only ids, to stay small. A feed needs names, pictures and text, so the service looks those up in a post cache and a user cache. This step is called hydration, filling a bare id with its full data. Media files come straight from a CDN (Level 1), not through the feed service.

Hydration

Hydration means replacing a list of ids with the full objects they point to, just before the answer is sent.

Example: a feed of 20 post ids becomes 20 posts with author name, picture, text and like count.

Check yourself: 1. Why does the web server check a post limit? 2. What does the feed cache hold? 3. Where do images come from?