CSE Roadmap

Core tier · about 4 hours · 11 parts

Part 3 of 11 · 2 min read

Splitting pays the shared background's cost once per worker, not once total

Every extra agent that needs the same context is an extra bill for that same context.

Think of a single chef reading one recipe card once. Compare that with handing a copy of the same long recipe card to five sous-chefs. Each of them has to read the whole thing before they can start their own piece of it.

In plain words: fan-out means dispatching independent sub-tasks to several agents at once. An orchestrator plans, dispatches and combines. Each worker executes one sub-task in its own context. The shared-context tax is the background every worker must separately load. Splitting multiplies that one-time cost by however many workers there are.

A worked example. With an 8,000-token system-plus-shared background and no prompt cache, splitting 8 independent sub-tasks uses only 0.45 of the single agent's total tokens. Isolated short contexts are cheap when nothing is cached. But sweep the shared background up from 0 to 20,000 tokens with a prompt cache active, and the token ratio climbs from 0.65 to 1.50. Past roughly 10,000 tokens of shared background, splitting costs more tokens than one agent with a cached long context. That is because every worker is re-paying for that same background from scratch.

The key lines. From code/mini-fanout/fanout.py:

# from code/mini-fanout/fanout.py
def multi(p: Params, deps):
    n = len(deps)
    L = levels(deps)
    tokens, ok = 0, 1.0
    for j in range(n):
        start = p.shared + p.brief + len(deps[j]) * p.result      # worker context: background, its brief, the results it depends on
        c, s, _ = run_agent(p, round(p.steps * (1 + p.redundancy)), p.tools_per_sub, start)
        tokens += c

Read it as: p.shared is added into start fresh, inside the loop, for every single worker j. There is no sharing of that cost across workers in this accounting. That is exactly what "the tax is paid once per worker rather than once" means. It's a line of code, not a metaphor.

Fan-out, orchestrator, worker

Fan-out is dispatching independent sub-tasks to several agents at once. The orchestrator is the agent that plans, dispatches and combines. A worker is an agent that executes one sub-task in its own context.

Example: the token ratio flipping from a 0.45 saving to a 1.50 loss as shared background grows is fan-out's shared-context tax made visible. The same structural choice, giving every worker its own copy of the background, makes isolation cheap for small backgrounds and expensive for large ones.