CSE Roadmap

Core tier · about 3 hours · 8 parts

Part 4 of 8 · 2 min read

Networks and slow processes

A call that never returns is worse than one that fails.

When a client sends a request and hears nothing, it has only two options: keep waiting, or give up. UDS lists four reasons for the silence:

  • the server is slow;
  • the request was dropped on the way;
  • the server crashed while working on it;
  • the reply was dropped on the way.

The client cannot tell which one. UDS calls slow network calls the silent killers of distributed systems, because a client that waits too long, or forever, causes problems that are very hard to trace.

Slow processes matter for the same reason. To an outside observer, a very slow process looks much like one that is not running. The common cause is a resource leak. A memory leak slowly fills memory, then the system swaps pages to disk and the garbage collector works harder, which slows everything more. Threads and connections leak too. If a thread makes a blocking call with no timeout and the call never returns, that thread is gone from its pool. The pool has a fixed size. Lose enough threads and none are left. Making the calls asynchronous does not fix it. Modern HTTP clients keep a pool of connections, and a request without a timeout never gives its connection back.

Libraries add risk. Code you did not write uses the same memory, threads and sockets, and you often cannot see how.

Resource leak

A resource leak happens when a program takes a resource, such as memory, a thread or a connection, and never gives it back.

Example: a request that waits forever holds one connection from a pool of 50. After 50 such requests, no connection is left.

Common mistake

Thinking that a slow service is better than a dead one. A dead service fails fast and callers move on. A slow one holds their threads and connections, and the slowness spreads to them.

Check yourself: 1. Name the four reasons a client may hear no reply. 2. Why does a missing timeout lead to a pool running dry? 3. Why does async calling not remove the problem?