Part 5 of 8 · 3 min read
Latency
Everything on one server.
Where latency comes from. The total is the time of each component on the path plus the network and disk time between them. Two ways to cut it: do less on the path, and remove network and disk waits.
Doing less. The critical path holds only what is needed. Even logging is taken off it.
Removing waits. Each network hop costs about 500 microseconds (a round trip). Several hops add up to milliseconds. Writing events to disk adds more, up to tens of milliseconds even with sequential writes. That was fine decades ago, but not now. Modern exchanges target tens of microseconds by putting the whole path on one server, so no network is involved. The components talk through mmap (memory-mapped files). A file in a memory-backed file system (/dev/shm) is mapped into the memory of each process, so a message is written once and read by others with no copy, no disk and no network. Sending takes under a microsecond.
Application loop. Each component runs a single thread in a tight loop, pinned to one CPU core. No other thread shares it, so there is:
- no context switch (the core is never taken away), and
- no lock, since one thread alone changes the state.
The cost is harder code: every task must be short, or it blocks the loop.
Event sourcing on the server. All components share one event store (an mmap file). Events (new order, order filled) are written once and read by every component. The state of each component is the result of applying the events (page 9). The gateway turns each message into a compact binary format and sends it as a "new order" event. The matching engine, with the order manager built in as a library, applies it. A fill is written back as an event. The market data publisher and the reporter read the same events.
The order manager is now a library embedded in each component that needs order states, since asking a central order manager would add latency. Each copy ends up in the same state because all apply the same events in the same order.
Single sequencer. The sequence number field is added by one sequencer that writes to the event store. Several writers would fight over the right to write and waste time on locks, so there is exactly one. It is simple and fast. Each component has a local ring buffer, and the sequencer pulls from them, stamps each event and appends it. Backup sequencers can wait to take over.
A worked example (made up). Compare two designs for one order. Separate servers with 6 network hops at 500 microseconds is 3 ms, plus a disk write of 5 ms: 8 ms. On one server, 6 mmap hops at 1 microsecond each is 6 microseconds, with the disk write moved off the path: about 10 microseconds in all. That is roughly 800 times faster.
Application loop
An application loop is a single thread, pinned to one core, that repeatedly polls for work and handles it, with no locks or switches.
Example: the order manager's thread checks the event store, handles one event, and checks again, never waiting for a lock.
Check yourself: 1. Why is the whole critical path on one server? 2. What does mmap give the components? 3. Why exactly one sequencer?