CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 2 of 7 · 1 min read

Purpose

A small store that everyone trusts.

Think of a notice board in the middle of a village. It holds only a few important notes: who is the mayor, who has the key to the well, which houses are open. Everyone reads it, and there is only ever one board.

What systems need from it.

  • One leader. A database partition, a job scheduler or a service of five replicas needs exactly one active member.
  • Mutual exclusion. One writer at a time for a file or a record.
  • Configuration. Settings that all instances must agree on.
  • Membership. Who is alive now?
  • Naming and discovery. Which servers hold which part of the service?
  • Work assignment. Which partition runs on which node?

Questions and assumptions here (made up).

  1. Clients? 100,000 client processes across many services.
  2. Data? A few hundred thousand small entries, under 1 GB in all. It fits in memory.
  3. Traffic? Mostly reads and watches. Writes are rare: locks, leader changes and configuration changes.
  4. Availability? The service must work if a minority of its nodes fail.
  5. Correctness? Never two holders of one lock, even when machines pause or networks split.

Key point. This is not a general database. It holds a small amount of critical data, and its job is to keep that data consistent. Clients rarely call it directly. Most use it through another system (such as a database or a streaming system that relies on it).

Coordination service

A coordination service is a small, replicated, strongly consistent store that gives programs locks, leader election, configuration and membership.

Example: five replicas of a service ask the coordination service who is leader, and only one gets the answer "you".

Check yourself: 1. Name three things systems need from a coordination service. 2. Why is the data small? 3. Why is it not used as a general database?