Part 3 of 8 · 1 min read
Numbers
Small data, strong guarantees.
All figures are made up to show the idea.
- Job definitions: 300,000.
- If every job ran each hour, launches per hour would be 300,000, which is about 83 a second on average. Real jobs run less often: assume 50,000 launches per hour on average, about 14 a second.
- Spikes: many people choose midnight or the start of an hour. Assume a peak of 20,000 launches in the one minute after midnight: 333 a second, 24 times the average.
- State per job: the definition (name, rule, command, owner), the next launch time, and a record per launch. 1 KB per job is 300 MB of definitions. A record of 200 bytes per launch at 50,000 launches an hour is 10 MB an hour, 240 MB a day, which is kept for days.
- Failover time: the election must finish in well under one minute, since the finest interval is one minute.
The finding: the data is small and the rate is modest. The difficulty is correctness: exactly the right launches, even when machines fail in the middle of work.
Check yourself: 1. How many launches a second is 50,000 an hour? 2. Why is the peak much higher than the average? 3. What is the hard part, if the rates are low?