CSE Roadmap

Common tier · about 3 hours · 8 parts

Part 1 of 8 · 1 min read

Why this page comes after warehouses

In this module you will

  • Explain encoding and decoding, and why built-in language formats are a poor choice for stored or shared data.
  • List the weak points of JSON, XML and CSV, and compare the byte sizes of the formats in a worked example.
  • Explain how Protocol Buffers use field numbers and how Avro uses a writer's and a reader's schema.
  • State the rules for adding, removing and renaming fields without breaking old or new code.
  • Plan a rolling change of a data format, including data that outlives the code.
  • Choose a format for a database, a service call and a data archive.

The previous pages decided where data lives and how it is found. Data also moves: from a server to a disk, from a service to another service, from a database to a warehouse. Each move turns data into bytes and back. In a real system, the old and new versions of code run at the same time, and the bytes written by one must be readable by the other. This is the reason many outages happen during a release. The tier is Common. The numbers on this page are made up to show the idea, unless the text names a book.