Part 3 of 8 · 3 min read
Hit, miss and the hit ratio
What happens the first time and every time after, and how to measure how well the CDN works.
Think of the local book shop again. The first buyer of a rare title makes the shop order it from the warehouse. The second buyer finds it on the shelf.
The book "System Design Interview" lists the steps for a file.
- A user requests
image.pngfrom a URL on the CDN's domain. - The edge server does not have the file in its cache. It asks the origin.
- The origin returns the file, with an optional header that says how long the copy may be kept (the TTL from page 1).
- The edge stores the file and returns it to the user. The copy stays until the TTL runs out.
- A second user asks for the same file.
- The edge answers from its copy, as long as the TTL has not run out.
Cache hit and cache miss
A cache is a store of saved copies for faster reuse. A hit is a request answered from the saved copy. A miss is a request where no valid copy is held, so the cache fetches from the origin.
Example: the first request for logo.png at an edge is a miss. The next thousand are hits.
Hit ratio
The hit ratio is the share of requests that are hits: hits ÷ (hits + misses).
Example: 950 hits and 50 misses give 950 ÷ 1,000 = 0.95, or 95 percent.
A worked example. A shop has 10 million page views a day. Each page pulls 2 megabytes of static files (images, scripts, styles).
- Total static data: 10,000,000 × 2 MB = 20,000,000 MB = 20 TB per day.
- Average rate: 20,000,000 MB ÷ 86,400 s ≈ 231 MB per second, or about 1,850 megabits per second.
- With no CDN, the origin must deliver all of it.
- With a CDN hit ratio of 95 percent, the origin delivers 5 percent: 20 TB × 0.05 = 1 TB per day, about 11.6 MB per second, or 93 megabits per second.
- The origin load drops by a factor of 20.
Raising the hit ratio from 95 to 99 percent cuts the origin load a further five times (1 percent against 5 percent). That is why hit ratio is the number to watch.
A hit ratio can be low for good reasons or bad ones. A file that few people request never gets a second hit. A cache key that splits one file into many copies (section 4) wastes hits. A very short TTL throws copies away too soon.
The book also names the cost: CDNs are run by third parties, and the owner pays for data moved in and out. Files that are rarely requested give little benefit, so they may be better left off the CDN.
Common mistake
Judging a CDN only by the average load time. A low hit ratio hides behind a good average if most users hit a few popular files. Look at the hit ratio by file type, and at the share of misses that go to the origin.
Check yourself:
- Out of 4,000 requests, 3,800 are hits. What is the hit ratio, and how many requests reach the origin?
- Why does a rarely requested file gain little from a CDN?