Part 3 of 8 · 1 min read
Load shedding
Reject the excess so the rest succeeds.
Load shedding
Load shedding means a server rejects some requests when it is overloaded, so it can finish the rest.
Example: a server with a limit of 200 concurrent requests answers the 201st with a 503 (Service Unavailable) status.
UDS says that on detecting overload the server fails fast and returns HTTP 503. It need not reject at random. If requests have different priorities, it can reject only the low-priority ones.
A worked example (made up). A shop's server can handle 200 concurrent requests. Requests are of three kinds: checkout (high), browse (medium), recommendations (low). At 180 concurrent, the server accepts all. At 200, it rejects recommendations first. At 230, it also rejects browse. Checkout is rejected last. Customers who are paying get through while others see a lighter page.
There is a limit to what shedding can do. Rejecting a request still costs something: the server may need to open a TLS connection and read the request before it says no. If the load keeps rising, the cost of rejecting takes over and the service degrades anyway.
Check yourself: 1. Which status code is the usual answer in load shedding? 2. In the example, which request kind is dropped first and why? 3. Why can shedding not save a service from unlimited load?