CSE Roadmap

Common tier · about 6 hours · 14 parts

Part 3 of 14 · 2 min read

Everything before the question runs again; put side effects after it

The server holds nothing between rounds, so the retry re-executes the tool body from the top -- including whatever it already did.

Think of a form that resets to the top every time you click "next" on one of its pages. If step one mailed a letter, clicking "next" again mails a second letter. The whole form reruns from the start.

In plain words: because the tool body runs again from the top on every round, anything written before the final question repeats on every round too. A side effect placed before the question happens once per round. The same side effect placed after the last question happens exactly once, which is the entire difference between a safe tool and a dangerous one.

A worked example. charge_bad logs a charge before asking "Charge X?" Measured, it ended with 2 side effects after a confirmed call, and 2 even when the person declined, because the log line ran on both rounds regardless of the answer. charge_good, identical except the log line moved to after the accept check, ended with exactly 1.

The key lines. From code/mini-input/servers.py:

# from code/mini-input/servers.py
def charge_bad(a, ctx):
    s.log.append(("charged", a["amount"]))                           # runs on every round
    r = ctx.need("ok", elicit_form(f"Charge {a['amount']}?", {"ok": {"type": "boolean"}}, ["ok"]))
    return "charged" if r["action"] == "accept" else "cancelled"

def charge_good(a, ctx):
    r = ctx.need("ok", elicit_form(f"Charge {a['amount']}?", {"ok": {"type": "boolean"}}, ["ok"]))
    if r["action"] != "accept":
        return "cancelled"
    s.log.append(("charged-once", a["amount"]))                      # only after the last question
    return "charged"

Read it as: the only difference between the two functions is which side of ctx.need the log line sits on. This is the whole bug and the whole fix in one place. charge_bad logs unconditionally before even knowing the answer, while charge_good only logs once it has already confirmed action == "accept", after the round that might repeat has already happened.

Multi round-trip request (MRTR)

A multi round-trip request is the modern mechanism: the call returns input_required instead of staying open, and the client resends the entire original call with the answers and an echoed requestState.

Example: charge_bad's doubled log entry is a direct, visible consequence of MRTR's own design. A protocol that reruns the tool body on every round will always double an unconditional side effect placed before the last question, in any tool, not just this one.