Part 4 of 7 · 3 min read
Numbers through the layers
How much traffic each layer removes, and what finally reaches the database.
Think of a series of filters in a water system. The first removes most of the dirt, the second removes most of what is left, the third a little more. What comes out of the last is a small share of what went in.
Each layer sees only the requests that the previous layer could not answer. The traffic reaching the database is the original traffic multiplied by the miss ratio of every layer.
A worked example. A page view causes a request for a product page. There are 100,000 requests in a minute.
| Layer | Hit ratio | Requests in | Answered | Passed on |
|---|---|---|---|---|
| Browser | 40 percent | 100,000 | 40,000 | 60,000 |
| CDN | 70 percent | 60,000 | 42,000 | 18,000 |
| Application in-process | 50 percent | 18,000 | 9,000 | 9,000 |
| Shared cache | 90 percent | 9,000 | 8,100 | 900 |
| Database | - | 900 | 900 | - |
Of 100,000 requests, 900 reach the database, which is 0.9 percent. The product of the miss ratios: 0.6 × 0.3 × 0.5 × 0.1 = 0.009. Each layer helps, but the first layers remove the most requests in absolute terms: 40,000 and 42,000 here.
Two lessons follow.
- Where a hit is cheapest, a small hit-ratio gain pays most. A browser hit costs no server work at all. Raising the browser hit ratio from 40 to 50 percent removes 10,000 requests from every later layer.
- Each extra layer helps less than the one before it. Remove the in-process layer: the shared cache sees 18,000 requests and passes 1,800 to the database. That is double (900 to 1,800). Whether an extra layer is worth its complexity depends on whether the database can take the difference.
Per-layer measurements matter. Log hit ratios per layer, and watch how many requests reach the database. A fall in one layer's ratio shifts load inward onto the next layer.
Latency adds up too. For a miss in every layer, the request pays each layer's check plus the database. For a hit at the browser, the cost is zero network time. The average latency of the example, with made-up times (browser 0 ms, CDN 30 ms, in-process 0.001 ms, shared 0.5 ms, database 20 ms, plus network to the data centre 40 ms):
- Browser hit (40,000 requests): 0 ms.
- CDN hit (42,000): 30 ms.
- Application in-process hit (9,000): 40 + 0.001 ≈ 40 ms.
- Shared cache hit (8,100): 40 + 0.5 ≈ 40.5 ms.
- Database (900): 40 + 0.5 + 20 = 60.5 ms.
- Average: (40,000 × 0 + 42,000 × 30 + 9,000 × 40 + 8,100 × 40.5 + 900 × 60.5) ÷ 100,000 = (0 + 1,260,000 + 360,000 + 328,050 + 54,450) ÷ 100,000 = 2,002,500 ÷ 100,000 ≈ 20 ms.
An average of 20 ms against 60.5 ms for every request going to the database. The browser and the CDN do most of the work.
Check yourself:
- 200,000 requests meet a CDN with a 75 percent hit ratio and then a shared cache with an 80 percent hit ratio. How many reach the database?
- Why is a browser hit worth more than a shared-cache hit?