CSE Roadmap

Core tier · about 5 hours · 8 parts

Part 2 of 8 · 2 min read

Scope and numbers

A drive that never loses a file.

Think of a safe-deposit room with a clerk at the door. Customers drop off boxes. Other branches have a copy of the list of what is in each box, so a customer can ask for any item from any branch. The room must never lose a box, and every branch must see the same list.

Questions and the answers assumed here (from the book's sample talk).

  1. Main features? Upload and download files, sync, notifications.
  2. Clients? Mobile and web.
  3. File types? Any.
  4. Encryption? Yes, files at rest are encrypted.
  5. File size limit? 10 GB.
  6. Users? 10 million daily active users (DAU).

Features in scope. Add files, download files, sync across devices, see file revisions, share files, and send notifications when a file is edited, deleted or shared. Real-time co-editing of a document is out of scope.

Requirements beyond features.

  • R1. Reliability. Data loss is not acceptable.
  • R2. Fast sync, or users leave.
  • R3. Low bandwidth use, especially on mobile data plans.
  • R4. Scalable and highly available.

Numbers (from the book). The service has 50 million registered users, of whom 10 million are daily active (DAU). Each user has 10 GB of free space. That is 50,000,000 × 10 GB = 500 PB allowed. Users upload 2 files a day of 500 KB on average. Upload requests per second: 10,000,000 × 2 / 86,400 = about 231, which the book rounds to 240. Peak is twice that, 480.

The key point: the request rate is small. The challenge is the volume of data, its safety and its consistency.

Sync

Sync is keeping the same files on many devices in step, so a change made on one appears on the others.

Example: a photo saved on a phone shows up in the laptop's folder a few seconds later.

Check yourself: 1. Why is the request rate not the main challenge? 2. What is the total allowed space? 3. Why is real-time co-editing left out?