Part 8 of 9 · 3 min read
Costs, limits and when to wait
The caveats the book lists, and the advice to start with a monolith.
Costs, limits and when to wait
Think of a revolving door at a building entrance. It controls who enters, but it is also the one place where everyone queues, and it needs maintenance.
The book states the drawbacks of an API gateway.
- A development bottleneck. Because it is coupled to the services it hides, every new service must be wired to it, and every API change in a service needs a change in the gateway.
- One more service to run. It must be developed, maintained and operated, and it must scale to the total request rate of all services behind it.
- A single point of failure. If the gateway is down, everything behind it is unreachable. It needs several copies and the care of page 3.
The book's view on balance: with dozens of services and APIs, the upside is greater than the downside and the gateway is a worthwhile investment. It suggests rolling one's own on top of a proxy framework such as NGINX, or better, using an off-the-shelf solution.
A worked example of load. Peak traffic across all services is 20,000 requests per second. Each public call makes 4 inner calls through the gateway's composition.
- Public calls: 20,000 per second arrive.
- If the gateway also handles the inner calls, 20,000 × 4 = 80,000 calls pass through it per second (this depends on the design: inner calls often go directly between services).
- A gateway node handles 15,000 public calls per second (made up). For 20,000, 2 nodes are enough, but 3 are needed to survive the loss of one (20,000 ÷ 2 = 10,000 each after a failure, under 15,000).
The book adds practical advice from the previous chapter: splitting an application into services adds a lot of complexity, so it is generally best to start with a monolith (one deployable unit) and peel off services when there is a good reason. A gateway is part of that complexity. A small team with one application does not need one; a simple balancer in front of the single application is enough. The gateway earns its place when:
- clients (mobile apps especially) call many services;
- the same cross-cutting jobs are repeated in many places;
- public APIs must stay stable while services change.
For the curious For the curious
Some teams run one gateway for each kind of client, such as one for mobile and one for web. Each is owned by the team that builds that client. This keeps the gateway close to the needs of its client and avoids one gateway that everyone must edit. It is called the backend for frontend pattern. It brings more gateways to run.
Interview angle
Answer in your head first, then open it.
How do services find each other?
A registry, with registration, heartbeats and expiry, or DNS for coarse needs. State the time a dead instance stays listed.
What does the API gateway do in your design?
Routing, composition for mobile, protocol translation, authentication, caching and rate limits. Keep business rules out of it.
What is the risk of composing many services?
Availability multiplies down. Mark calls as required or optional, use parallel calls, time limits and fallbacks.
Do you need a gateway on day one?
Not for a small monolith. Add it as the number of services and clients grows.
Check yourself:
- Name two costs of an API gateway.
- When is a gateway not worth building?
Words you will meet
| Word | Plain meaning |
|---|---|
| Service discovery | Finding the current addresses of a service |
| Service registry | A live list of running instances of each service |
| Heartbeat | A small regular message that says "still alive" |
| Server-side, client-side discovery | The balancer looks up, or the caller looks up |
| API gateway | One public door that routes calls to internal services |
| Routing map | The table linking public paths to internal ones |
| Composition | Merging replies from several services into one |
| Fan-out | The number of inner calls caused by one outer call |
| Authentication | Checking who the caller is |
| Authorization | Deciding what the caller may do |
| Opaque, transparent token | A token with no facts inside, and one with facts and a signature |
| API key | A secret string that identifies an application |
| Monolith | An application deployed as one unit |