Part 4 of 14 · 2 min read
A dropped connection after the work is done needs a key, not a retry
Think of mailing a letter at the exact moment the postal worker's receipt machine jams: you have no way to tell, from where you stand, whether the letter actually went out.
In plain words: when a connection dies after a server has run a tool but before the reply arrives, the client cannot tell "never arrived" from "it happened and I never heard." A read-only call can always be retried safely. A call with a side effect can only be retried safely if the server can recognise a repeat, which is exactly what an idempotency key is for.
A worked example. The example's server runs place_hold and then deliberately cuts the connection before replying. A client that simply retries the same call ends up with 2 attempts and 2 holds: a duplicate. A client that retries with an idempotency key attached ends up with 2 attempts and only 1 hold instead, because the server recognises the second call as a repeat of the first and returns the same result marked as a repeat. A client retrying a read-only search gets the same result both times regardless, because nothing about a read can duplicate.
The key lines. From code/mini-transport/servers.py:
# from code/mini-transport/servers.py
def place_hold(a):
"""Side effect, not safe to repeat: two calls make two holds."""
if a["isbn"] not in books:
raise ToolFailure(f"No book with isbn {a['isbn']!r}. Use search to find the isbn.")
key = a.get("idempotency_key")
if key and key in seen_keys:
return {**seen_keys[key], "repeat": True}
hold = {"hold_id": f"h{len(s.holds) + 1}", "isbn": a["isbn"], "member": a["member"]}
Read it as: the function's own docstring states the danger plainly, and the fix is one dictionary lookup. seen_keys remembers every key it has already handled. So a second call carrying the same key never creates a second hold. The key is generated once per intended action. It is what lets the server tell "this is a retry" from "this is a new request that happens to look similar."
Idempotency key
A unique value generated once for each intended action and sent with every attempt to perform it; a server that remembers results by key can recognise a repeated call and return the first result instead of repeating the effect.
Example: without a key, there is no honest way to answer "did my hold go through" after a dropped connection except to call list_holds and check. That is the same rule module 1 gives for anything that was not retried safely: tell the model the outcome is uncertain and name the next step.