Part 5 of 6 · 4 min read
Scheduling
Something must decide which machine runs which container.
Scheduling and orchestration
BM argues for a scheduling layer: you ask for a container to run, and the layer picks a machine that can run it. It names Google's recently released Kubernetes and CoreOS's cluster technology as early examples, and notes that new entrants appear every month. It places Docker plus a scheduling layer between infrastructure-as-a-service (rented machines) and platform-as-a-service (a hosted environment), a term it calls containers as a service.
What the books say ends there. The rest of this section is common practice, not from the books, and describes what an orchestrator such as Kubernetes does today:
- Desired state. You declare "run 5 copies of checkout, image version X". The orchestrator keeps making it true. If a machine dies and 2 copies vanish, it starts 2 more elsewhere. This is the self-healing of Level 8 module 8.
- Placement. It chooses a machine with enough free CPU and memory for each container, and can spread copies across machines or zones.
- Rolling updates. It replaces copies in small batches and can stop and go back, which matches the staged rollout of module 4.
- Health checks. It restarts a container that fails its liveness check and withholds traffic from one that fails its readiness check (Level 8).
- Service discovery and load balancing. It gives a group of copies one stable name and spreads requests across them.
- Scaling. It can add or remove copies based on a metric such as CPU or request rate.
- Config and secrets. It supplies settings and secret values to containers at start-up, so they are not baked into the image (module 7).
Common Kubernetes words (common practice): a pod is the smallest unit, one or more containers scheduled together. A deployment says how many pods of what image to keep running. A service is a stable name in front of a set of pods. A node is a machine in the cluster.
A made-up example. The team declares 6 pods of checkout across 3 nodes. A node fails at 14:00. The orchestrator sees 4 pods left, starts 2 on the other nodes within about a minute, and traffic is never sent to the dead pods because their health checks fail.
Cost (common practice): an orchestrator is a large system to learn and run. For a handful of services a managed platform or plain VMs may be simpler. BM makes a similar point elsewhere about adding technology only as needed.
Orchestrator
An orchestrator is software that runs a cluster of machines as one pool. It places containers, restarts failed ones, updates them in steps and scales them to match a declared desired state.
Example: told "keep 6 copies of checkout running", it starts a 6th when one crashes.
Common mistake
Treating containers as safe to run untrusted code, or storing important data inside a container's own disk. A container can be killed and rebuilt at any moment, and its disk goes with it. Keep state in a database or a volume outside it (common practice).
Interview angle
Answer in your head first, then open it.
Container or VM?
Containers share the host kernel, start in seconds and pack densely. VMs have their own operating system and give stronger isolation.
What does Kubernetes do?
It schedules containers onto machines and keeps the declared number running, with rolling updates, health checks, discovery and scaling.
Why not run everything in one container?
One service per container keeps deployment and failure small and independent.
Is a container a security boundary?
Not a complete one. Untrusted code needs stronger isolation.
Check yourself: 1. What does "desired state" mean? 2. What happens when a node dies? 3. Name one reason not to adopt an orchestrator for two services.
Words you will meet
| Word | Meaning |
|---|---|
| Virtual machine | A simulated computer with its own operating system |
| Hypervisor | The layer that divides hardware among VMs |
| Container | An isolated process group sharing the host kernel |
| Image | A frozen, versioned package of a service and its needs |
| Registry | A store of images by name and version |
| Scheduler | The part that decides which machine runs which container |
| Orchestrator | Software that runs a cluster to a declared desired state |
| Pod | The smallest Kubernetes unit: containers scheduled together |
| Node | A machine in the cluster |