CSE Roadmap

Common tier · about 4 hours · 8 parts

Part 2 of 8 · 2 min read

Scope

Bring content close to the user.

Think of a chain of shops. Instead of every customer travelling to the one factory, each shop keeps stock of popular items. The factory is the origin, the shops are the edge servers.

What a CDN is. A network of servers spread around the world that cache static content such as images, videos, CSS and JavaScript. When a user asks for a file, a server near the user answers. The further the user is from the server, the slower the page loads.

The basic flow (from the book).

  1. The user requests a file by a URL whose domain belongs to the CDN.
  2. If the nearby CDN server has no copy, it requests it from the origin (a web server or storage such as an object store).
  3. The origin returns the file, with an optional header giving a time to live (TTL): how long it may be cached.
  4. The CDN caches the file and returns it.
  5. Another user asks for the same file. The CDN returns it from the cache while the TTL has not expired.

Questions and assumptions here (made up).

  1. Content? Static files: images, scripts, video segments. Dynamic caching is out of scope.
  2. Traffic? 2 trillion requests a month.
  3. Locations? 150 edge locations (points of presence, PoPs) in 60 countries.
  4. Latency goal? The first byte within 50 ms for most users, with a high cache hit ratio.
  5. Reliability? One PoP or one origin failure must not take the site down.

Numbers (made up).

  • Requests: 2,000,000,000,000 a month / (30 × 86,400) is about 770,000 requests a second on average. Peak 3 times: 2.3 million a second.
  • Average object size 100 KB. Average traffic is 770,000 × 100 KB = 77 GB/s, which is 616 Gbps. Peak at 3 times is about 1.85 Tbps for this mix.
  • Per PoP: 2,300,000 / 150 = 15,000 requests a second on average at peak, spread unevenly.
  • Hit ratio goal: 95 per cent. Then only 5 per cent of requests reach the origin: 770,000 × 0.05 = 38,500 a second on average. At 99 per cent, only 7,700.
  • Cache size: the hot set per PoP, say 20 TB of disk plus 512 GB of memory.

Findings: the hit ratio is the key number. Each point of hit ratio removes load from the origin and cuts latency and cost.

CDN

A content delivery network is a set of cache servers around the world that serve copies of content from a place near the user.

Example: a user in Mumbai gets a logo from a server in Mumbai instead of from an origin in Virginia.

Check yourself: 1. What does the origin do? 2. What does the TTL control? 3. Why is the hit ratio the key number?