Part 6 of 8 · 1 min read
Myths and mistakes
Common wrong beliefs that cause outages.
This section is common practice, not from the books used here. It lists beliefs that lead to wrong numbers. Each is linked to a point above.
| Myth or mistake | The truth | See |
|---|---|---|
| "Average latency is 80 ms, so all is well" | The mean hides the tail | Section 2 |
| "CPU is at 60 percent, so load can double" | Another resource (connections, locks, a downstream service) may reach its limit first | Section 1 |
| "Our test passed at 500 users" | A closed-loop test cannot overload the system | Section 3 |
| "We tested on a small data set" | Caches, indexes and query plans change with size | Section 3 |
| "Autoscaling will handle the spike" | It reacts after the spike has started | Section 4 |
| "We have 10 servers, so capacity is 10 times one" | Shared parts (database, cache, network) do not scale with the web tier | Section 1 |
| "A failure test passes, so the system is resilient" | Tests find only failures someone imagined (UDS) | Section 6 |
Common mistake
Treating the result of one test as a permanent fact. Code, data size and traffic mix change. A capacity figure older than a few releases should be tested again.
Check yourself: 1. Why can CPU at 60 percent still be a problem? 2. Why does a shared database break "10 times one"? 3. Why should capacity numbers expire?