CSE Roadmap

Common tier · about 3 hours · 8 parts

Part 6 of 8 · 1 min read

Myths and mistakes

Common wrong beliefs that cause outages.

This section is common practice, not from the books used here. It lists beliefs that lead to wrong numbers. Each is linked to a point above.

Myth or mistake The truth See
"Average latency is 80 ms, so all is well" The mean hides the tail Section 2
"CPU is at 60 percent, so load can double" Another resource (connections, locks, a downstream service) may reach its limit first Section 1
"Our test passed at 500 users" A closed-loop test cannot overload the system Section 3
"We tested on a small data set" Caches, indexes and query plans change with size Section 3
"Autoscaling will handle the spike" It reacts after the spike has started Section 4
"We have 10 servers, so capacity is 10 times one" Shared parts (database, cache, network) do not scale with the web tier Section 1
"A failure test passes, so the system is resilient" Tests find only failures someone imagined (UDS) Section 6

Common mistake

Treating the result of one test as a permanent fact. Code, data size and traffic mix change. A capacity figure older than a few releases should be tested again.

Check yourself: 1. Why can CPU at 60 percent still be a problem? 2. Why does a shared database break "10 times one"? 3. Why should capacity numbers expire?