CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 7 of 8 · 4 min read

Origin shielding, stampedes and failures

Protecting the origin from a crowd of misses, and planning for the day the CDN fails.

Origin shielding, stampedes and failures

Think of a thousand local shops all running out of the same book at the same time. If each one calls the warehouse, the warehouse phone melts. If the shops call a regional depot, and only the depot calls the warehouse, the warehouse takes one call.

This is origin shielding.

Origin shield

An origin shield is a middle layer of CDN servers between the edge servers and the origin. Edges that miss ask the shield. Only the shield asks the origin.

Example: 200 edges all miss on a new video. The shield makes one request to the origin and answers the other 199 from its copy.

A worked example. A file is new, or has just been purged, and 200 edges must fetch it.

  • Without a shield: up to 200 requests to the origin, one from each edge.
  • With one shield: 1 request.
  • With 3 regional shields: 3 requests.

The same herd problem exists inside one edge. If a popular file expires, and 5,000 users ask for it in the same second, an edge that treats each as a separate miss sends 5,000 requests to the origin. This is a stampede. The cure is request collapsing: the edge sends one request, holds the others for a moment, and answers all of them from the result. Staggered TTLs, a small random amount added to each, also stop all copies from expiring together.

The book names two more risks.

  • CDN fallback. If the CDN has an outage, clients should detect the problem and request the resources from the origin. The origin must be able to carry that load, at least for the essential files, or the site should degrade gracefully (for example, text without images). Many sites keep a second CDN or an alternative host name ready.
  • Cost. Data in and out is charged. Rarely used files may not repay the cost.

Other points an interviewer may probe:

  • Large files such as video are cut into small pieces (segments) of a few seconds, so each piece is cached on its own and playback starts without the whole file. Level 10 comes back to this.
  • Security. Signed URLs, which contain an expiry time and a signature, let the CDN serve private files only to users who got a valid link.
  • Uploads go the other way. Users upload to a nearby entry point, which forwards to storage.
  • Edge compute. Some CDNs run small programs at the edge, for example to rewrite a URL, check a token or pick an experiment variant. This pushes some dynamic work closer to the user.

Interview angle

Answer in your head first, then open it.

Where would you use a CDN in this design?

Static assets and media, with long TTLs and versioned names. Name the cache key and say what is left out of it.

A file was changed but users still see the old one. What do you do?

Check the TTL and the key, purge if urgent, and move to versioned names so the problem does not return.

What happens if the CDN goes down?

Clients fall back to the origin, which must be sized for the essential files, or a second CDN takes over.

How do you stop a popular file expiring everywhere at once?

Add a random amount to each TTL, collapse duplicate misses at the edge, and use an origin shield.

Check yourself:

  • With 120 edges and 2 shields, how many requests reach the origin for a brand-new file?
  • What is a stampede, and what are two cures?

Words you will meet

Word Plain meaning
CDN Servers around the world that hold copies of files near users
Origin The server that owns the real file
Edge server A CDN server near users
Cache A store of saved copies for faster reuse
Hit, miss A request answered from the saved copy, and one that is not
Hit ratio Hits divided by all requests
Static, dynamic The same for everyone, and built per request
Cache key The values that must match for two requests to share a copy
Cache-Control A header that says how long and where a reply may be saved
ETag A label for one exact version of a file
304 Not Modified A reply with no body: the saved copy is still good
Vary A header that lists the request headers that change the reply
Purge An order to every edge to drop its copy
Origin shield A middle layer that protects the origin from many misses
Stampede Many requests for one expired item at the same instant