CSE Roadmap

Core tier · about 6 hours · 7 parts

Part 4 of 7 · 3 min read

Registry.execute never raises

Every kind of bad request becomes an error message the model can read, and why model output is untrusted input.

The most important function in this module is Registry.execute. It validates, runs, and turns every problem into text for the model, and it never raises:

Problem Result sent back to the model
Tool name does not exist Error: unknown tool 'get_forecast'. Available tools: get_weather, calculate.
Arguments are not valid JSON Error: arguments are not valid JSON (Expecting value).
Arguments are JSON but not an object Error: arguments must be a JSON object such as {"city": "Pune"}, not an array.
Arguments fail the schema Error: invalid arguments: city: Field required.
An argument the tool does not know Error: invalid arguments: units: Extra inputs are not permitted.
The function throws Error: the tool failed (KeyError: 'no data for Atlantis').
The result cannot become text Error: the tool failed (...)
Result is huge The first part, plus ... [truncated, 3000 more characters]

The demo shows the payoff. For each of three wrong first requests, the model reads the error and corrects itself on the next step, and the run still ends with an answer. A crashed loop can only be fixed by you. An error message can be fixed by the model.

Good error messages say what was wrong and what is allowed. They are prompts too.

Validation is a security boundary

Model output is untrusted input, exactly like a web form. Always validate arguments against the schema before running anything. Never trust the model to stay within the rules you wrote in the prompt (module 1). The tool itself should also enforce limits, such as allowed folders for a file tool, allowed domains for a fetch tool, and maximum amounts for a payment tool.

The example's calculator shows the pattern. It could have used Python's eval, which would let a model (or text that influences it) run arbitrary code. Instead it parses the expression and allows only numbers and arithmetic operators. Tests confirm that __import__('os').system('ls'), open('x'), a + 1 and [1][0] are all refused.

Never eval model text

This is the single most common serious mistake in early agents. The same rule applies to shell commands, SQL, file paths and URLs: validate against an allow-list, or run them in a sandbox (Level 10).

For the curious Two bugs the first version had

1. It did raise. The old notes said execute never raises, but json.dumps(result) sat outside the try. The expression (10**100)**100 computed fine and then crashed the whole loop with ValueError: Exceeds the limit (4300 digits). A function returning a set would crash it too. The conversion to text is now inside the try.

2. The calculator could freeze the agent. It capped each exponent at 100, which looks safe. But (((9**99)**99)**99)**99 is four small exponents that multiply into an enormous number. ((9**99)**99)**99 took 0.24 s; one more **99 ran for over 60 seconds before I stopped it. A model, or text steering it, could freeze your agent with an input of under 25 characters. The calculator now refuses long expressions (over 200 characters), refuses any integer above 4,096 bits before computing a power, and rejects results that are not finite real numbers (1e999 * 1e999, (-8)**0.5) or booleans (True + 1). Lesson: a safe parser is not enough. A tool also needs resource limits.

Check yourself

Why is returning Error: ... as a normal tool result better than raising an exception? (The model can read the text and try again. An exception stops your loop, and only you can fix it.)