CSE Roadmap

Core tier · about 5 hours · 8 parts

Part 6 of 8 · 2 min read

Notifications and conflicts

Telling devices, and settling clashes.

Notification service. It lets the server send events to clients as they happen. Two options: long polling (Level 6, used by Dropbox) and WebSocket (module 2). The book chooses long polling. The traffic only goes one way, from server to client. Updates are rare and come in no big bursts, so a two-way channel is not worth it. Each client holds a long poll connection. When a change appears, the connection closes with an answer, and the client fetches the changes, then opens a new connection at once.

Sync conflicts. A conflict arises when two users change the same file at nearly the same time. The book's rule is that the version processed first wins. The second user gets a conflict. The system shows both copies, the local one and the latest from the server, and the user can merge them or choose one.

A worked example (made up). Anna and Ben both edit report.doc offline. Anna syncs first, so her change is saved as version 5. Ben syncs next and sees a conflict. He keeps both files, or merges by hand, and the result becomes version 6.

Saving space. Versions kept in several data centres fill storage quickly. Three techniques:

  • De-duplication. Blocks with the same hash are identical, so keep only one copy per account.
  • Limit versions. Cap the number of stored versions, and favour recent ones. A document saved a thousand times in a short span need not keep all copies.
  • Cold storage. Data not used for months or years moves to cheaper storage such as S3 Glacier.

A worked example (made up). A 50 MB file is uploaded by 1,000 people in the same account group. Without de-duplication, 1,000 × 50 MB = 50,000 MB (about 50 GB) is stored. With it, 50 MB is stored once, a saving of 49,950 MB. (The book de-duplicates at the account level, so the real gain depends on how the files are shared.)

De-duplication

De-duplication means storing identical data once, and pointing every owner to that single copy.

Example: two blocks with the same hash are the same content, so storage keeps one block and two references.

Check yourself: 1. Why does the book pick long polling here? 2. Who wins a conflict, and what does the loser see? 3. What does a hash tell you about two blocks?