CSE Roadmap

Common tier · about 4 hours · 8 parts

Part 3 of 8 · 1 min read

Numbers

Small data, strong guarantees.

All figures are made up to show the idea.

  • Job definitions: 300,000.
  • If every job ran each hour, launches per hour would be 300,000, which is about 83 a second on average. Real jobs run less often: assume 50,000 launches per hour on average, about 14 a second.
  • Spikes: many people choose midnight or the start of an hour. Assume a peak of 20,000 launches in the one minute after midnight: 333 a second, 24 times the average.
  • State per job: the definition (name, rule, command, owner), the next launch time, and a record per launch. 1 KB per job is 300 MB of definitions. A record of 200 bytes per launch at 50,000 launches an hour is 10 MB an hour, 240 MB a day, which is kept for days.
  • Failover time: the election must finish in well under one minute, since the finest interval is one minute.

The finding: the data is small and the rate is modest. The difficulty is correctness: exactly the right launches, even when machines fail in the middle of work.

Check yourself: 1. How many launches a second is 50,000 an hour? 2. Why is the peak much higher than the average? 3. What is the hard part, if the rates are low?