Part 2 of 12 · 2 min read
Why a web request is the wrong shape for a 10-minute job
The restaurant-ticket fix for a run that outlives one HTTP connection.
Think of a busy restaurant. You do not stand at the counter until your food is ready; you get a ticket number, sit down, and check back. A long agent run needs the same thing: submit, get a number, check later.
In plain words: a single HTTP request that blocks until a 10-minute agent run finishes will usually die before the run does. Browsers, proxies and load balancers all close connections that sit idle too long.
A worked example. 1,000 requests, each a run that takes 3 minutes on average, against a proxy that times out after 2 minutes. If nothing retries, only the runs that happen to finish inside 2 minutes are answered: 459 of 1,000. If the client retries once on timeout, more get answered: 715. But the server now has two copies of every run that timed out. It did 1,541 runs for 1,000 requests, and 541 of them were wasted.
The fix is to split "accept the work" from "do the work." POST /runs returns at once with a ticket (a job id). The client checks status later, or is notified.
Job
A record of one requested run, with a state, a result and a cost, stored by the server.
Example: {"id": "j-7f2a", "owner": "acme", "state": "running", "result": None}.
The key lines. Here is the whole submit step, from code/mini-jobapi/jobapi.py:
# from code/mini-jobapi/jobapi.py
def submit(self, owner, payload, idem_key=None, now=0.0):
"""Returns (http_status, job). 202 for new, 200 for a replay, 409 for key reuse with a different body."""
...
return 202, j
Read it as: submit never runs the agent. It only creates a record and returns 202 at once. The agent runs elsewhere, in a worker (module 2).