Part 4 of 12 · 2 min read
Pause the run instead of blocking the worker on a slow answer
Think of a chef who does not stand at the pass waiting for a single table's special request to come back from the dining room. Instead, the chef starts the next dish. Then the chef comes back to the first one the moment the answer arrives.
In plain words: an interrupt stops a run at a point and saves it to wait for input, and resume continues from that saved state once the person's answer arrives -- interrupting and saving durably lets the worker move on to other tasks while waiting, instead of sitting idle, which matters because waits for a human decision are often much longer than the actual work.
A worked example. With 50 tasks, each needing 5 minutes of actual work and a mean wait of 30 minutes for a person's decision, a worker that blocks (sits idle during each wait) takes 1,663 minutes to finish all 50. A worker that pauses the run (checkpoints the state, frees itself, and picks up other ready tasks) finishes the same 50 tasks in just 272 minutes -- more than 6 times faster, purely from not wasting worker-time on waiting. At a 120-minute mean wait the gap widens further: 5,904 minutes blocking against 713 pausing.
The key lines. From code/mini-hitl/hitl.py:
# from code/mini-hitl/hitl.py
if mode == "blocking":
return sum(before + w + after for w in waits)
t = 0.0
ready = [] # (ready_time, kind, id)
for i in range(n):
heapq.heappush(ready, (0.0, 0, i))
Read it as: the blocking branch is a one-line sum over every task's own wait. The pausing branch builds a priority queue (ready) that lets the worker jump to whichever task is actually ready next. The blocking calculation is the worker's total idle time added up. The queue-based calculation is what a worker that never sits idle actually achieves. The gap between the two numbers is the entire cost of not building durable pause-and-resume.
Interrupt, resume
An interrupt is stopping a run at a point and saving it to wait for input. Resume is continuing from the saved state with the person's answer.
Example: 1,663 minutes versus 272 minutes for the same 50 tasks is interrupt-and-resume's entire value proposition in two numbers -- the work didn't get any faster, the waiting simply stopped being charged to the worker's clock.