Part 2 of 8 · 2 min read
Scope
Typing together, seeing each other.
Think of two people writing on one sheet of paper. If each writes in a different spot, nothing clashes. If both write at the same spot, a rule must say whose words come first, and both must end with the same page.
Questions and assumptions here (made up).
- Content? Rich text documents, up to about 100 pages.
- Collaborators? Up to 100 editors at once on one document, many more viewers.
- Features? Live typing, cursors and selections of others (presence), comments, version history, offline editing.
- Latency? A change should appear on other screens within about 200 ms. Own typing appears at once.
- Correctness? All copies must end up identical, and no one's text may be lost.
- Scale? 100 million documents, 20 million active a day.
Numbers (made up).
- Active documents at one moment: 2 million.
- Most have 1 editor. Say 5% have 2 or more: 100,000 documents with several editors.
- One typist makes about 3 changes a second (batches of keystrokes). With 4 million active editors typing at times, say 1 in 4 typing at a moment: 1 million editors × 3 = 3 million changes a second at peak. Treat as an upper estimate; with typing pauses assume 1 million a second.
- Each change is small: 100 bytes. 1,000,000 × 100 B = 100 MB/s into the system.
- Fan-out: for a document with 5 editors, one change goes to 4 others. Average fan-out about 1.5 across all documents.
- Storage: a document with 50 KB of text and a history of changes of 20× the size is about 1 MB. 100 million × 1 MB = 100 TB.
Findings: writes per document are low (a few a second), but there are very many documents, so the key to scale is partition by document. Each document is handled by one server at a time.