CSE Roadmap

Core tier · about 3 hours · 7 parts

Part 3 of 7 · 3 min read

Reading your own writes

The case where a user saves something and then cannot see it.

Think of posting a letter and walking to the sorting office the next minute to ask whether it arrived. The clerk checks the shelf where the mail from your area is not yet filed, and says there is nothing. You worry that it was lost. In fact it just has not reached that shelf.

Many applications let a user submit data and then view it: a profile change, a comment, a post. The write goes to the leader. The view can be served from a follower. With asynchronous replication the follower may not have the write yet, so the user sees the old page and thinks the data was lost.

Read-your-writes consistency

Read-your-writes (also called read-after-write) consistency is the guarantee that after a user saves something, any later read by the same user shows it. It makes no promise about other users, who may see the change a little later.

Example: a user edits a display name and reloads the page. The new name must show, though a friend may still see the old name for a few seconds.

The book lists ways to achieve it with leader-based replication.

  1. Read what the user may have changed from the leader. Read everything else from a follower. This needs a rule that tells which data the user may have changed. On a social network, a profile is edited only by its owner, so the simple rule is: always read your own profile from the leader and other people's profiles from a follower.
  2. Use time since the last write. If most data is editable by the user, the first rule sends nearly all reads to the leader and loses the benefit of read scaling. Instead, for one minute after the user's last update, read everything from the leader. Or watch the lag of the followers and block reads on any follower more than a minute behind.
  3. Remember the position of the last write. The client keeps a timestamp of its most recent write. The system ensures the follower serving a read has applied changes at least up to that time. If the follower is behind, another replica answers, or the read waits. The timestamp can be a logical one, such as the log sequence number (module 1), or a real clock time, in which case clock synchronisation matters.
  4. Several data centres. A request that needs the leader must be routed to the data centre that holds the leader.

Cross-device read-your-writes adds more difficulty. If a user types on a laptop and looks on a phone, the phone should see the change. The timestamp of the last update is now needed from both devices, so that information must be kept centrally. If the devices connect through different networks, requests may land in different data centres. If the approach needs the leader, all of a user's devices must be routed to the same data centre.

A worked example of the cost. A site has 50,000 reads per second. 10 percent of reads are by a user within one minute of that user's own write (made up).

  • Rule 2 sends 10 percent of reads to the leader: 5,000 reads per second.
  • The other 45,000 go to followers. The leader's read load is 5,000 plus its writes.
  • If the leader's capacity is 10,000 operations per second and the writes are 2,000 per second, the leader runs at 7,000. That fits.
  • If 40 percent of reads were recent-write reads, the leader would get 20,000 reads per second and fail. The rule only works when the share is small.

Common mistake

Sending every read to a random follower, and then adding a refresh button for users who think their data was lost. The guarantee should be built in, for the data users have just changed.

Check yourself:

  • A site sends reads to the leader for 2 minutes after a user's write. Users write once every 10 minutes on average. About what share of a user's reads go to the leader?
  • Why is cross-device read-your-writes harder than single-device?