CSE Roadmap

Common tier · about 4 hours · 11 parts

Part 2 of 11 · 2 min read

A claim of "done" needs a checker that is not the agent itself

A builder who says the job is finished is making a claim, not reporting a fact, and the two look identical from the outside.

Think of a builder announcing a job finished. A sensible homeowner does not just believe it. They call an inspector to check the work against the actual plan before paying the final invoice.

In plain words: success criteria are conditions that define when the goal is met, written so they can be checked mechanically or by a rubric, a verifier is a check independent of the agent's own judgement, and a claim is simply the agent saying the task is done -- treating that claim as a hypothesis to verify, rather than a fact to accept, is the entire difference between a system that can be fooled and one that can't.

A worked example. Letting the agent's own claim end the run produces a false "done" on 40% of tasks, once the agent over-claims at just 5% per step after reaching three of five subgoals. Adding an independent verifier that checks the claim before accepting it cuts that false-completion rate to 0.9% -- nearly a 44x reduction, for exactly the same underlying agent behaviour. The only thing that changed is whether something other than the agent gets a vote on whether "done" is true.

The key lines. From code/mini-goals/goals.py:

# from code/mini-goals/goals.py
claim = (complete and r.random() < p.claim_true) or (not complete and done >= 3 and r.random() < p.overclaim)
if claim:
    if not verified:
        return {"outcome": "success" if complete else "false_done", "steps": steps, "cost": cost, "infeasible": infeasible}
    if complete or r.random() < p.verifier_fp:
        return {"outcome": "success" if complete else "false_done", "steps": steps, "cost": cost, "infeasible": infeasible}

Read it as: when verified is false, any claim ends the run immediately as either success or false_done. The code never checks complete again before returning. When verified is true, an incomplete claim only slips through via p.verifier_fp, the verifier's own false-positive rate. The entire safety gain comes from inserting one extra condition between "agent claims done" and "run actually ends."

Success criteria, verifier, claim

Success criteria are conditions that define when the goal is met, written so they can be checked mechanically (tests pass, a field is present, a number is within a range) or by a rubric. A verifier is a check that is independent of the agent's own judgement: a test suite, a validator, a second model with a rubric, or a person. A claim is the agent saying that the task is done.

Example: 40% false completions falling to 0.9% is what happens when "the agent said so" (a claim) is replaced by "something else checked it" (a verifier) -- the fix isn't a smarter agent, it's a structural one: the entity deciding success is no longer the entity that might be wrong.