CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 2 of 8 · 1 min read

Why say no

A busy server does not fail cleanly, it crawls.

Think of a restaurant with 20 tables. If staff keep seating guests beyond 20, nobody is served well. The kitchen jams, food goes cold, and even the first guests leave unhappy. A good host says "the wait is 30 minutes" or "the house is full."

UDS describes the same for servers. The operating system keeps a limited queue of connections per port, and when it is full new attempts are refused at once. But under heavy load, a server usually slows to a crawl before that limit is reached. It runs out of memory, threads, sockets or file handles, and its response time grows until it is unavailable to the outside world.

When a server is at capacity there is no good reason to take more requests. Excess requests only degrade it further. It should refuse them so it can finish the ones it has.

Overload

A service is overloaded when it receives more work than it can finish at an acceptable speed.

Example: a service that can handle 200 requests at once is given 400. Each takes twice as long, then four times as long, and callers begin to time out.

UDS says the meaning of overload depends on your system, but it should be measurable and actionable. A good measure is the number of requests being handled at once: add 1 to a counter when a request arrives and subtract 1 when the answer is sent.

Check yourself: 1. Why does a server often become unavailable before the operating system refuses connections? 2. What simple measure does UDS suggest for overload? 3. Why is it better to refuse extra requests than to accept them?