CSE Roadmap

Core tier · about 4 hours · 11 parts

Part 5 of 11 · 3 min read

What the example found

Latency and tokens by dependency density, the sub-task-count and shared-background sweeps, and reliability under two hazard settings.

Setup: 8 sub-tasks (6 steps each) unless stated. The averages are over 20 random dependency graphs. One dependency edge to each of the three preceding sub-tasks is present with probability equal to the density.

How to read the density table: each row is a different dependency density, from fully independent (0) to a full chain (1.0), without a prompt cache.

Density Levels Edges Tokens, split / single Latency, split / single Success, split vs single (weak model)
0 1.0 0 0.45 0.23 0.71 vs 0.33
0.2 2.9 3.4 0.46 0.54 0.65 vs 0.33
0.5 5.2 9.4 0.47 0.97 0.53 vs 0.33
0.8 6.8 14.6 0.48 1.24 0.46 vs 0.33
1.0 8.0 18.0 0.49 1.46 0.41 vs 0.33

What to notice: the token ratio barely moves across the whole density sweep (0.45 to 0.49). The latency ratio, though, swings from a 77% saving to a 46% loss. Density is almost entirely a latency story here, not a token story.

How to read the sub-task-count table: each column is a different number of sub-tasks at fixed density 0.1; rows compare with and without a prompt cache.

Sub-tasks 2 3 4 8 12
no prompt cache 1.15 0.91 0.76 0.46 0.33
with prompt cache (read at 0.1) 1.44 1.33 1.21 0.87 0.67

What to notice: splitting only two sub-tasks costs more tokens than one agent in both settings (1.15 and 1.44). The fixed overhead of a second system prompt and a copy of the background isn't worth paying until there are enough sub-tasks to spread it across.

How to read the shared-background table: each pair of numbers is the token ratio without and with a prompt cache, at a given shared-background size (8 sub-tasks, density 0.1).

Shared background 0 tokens: 0.31 without a cache, 0.65 with. 3,000: 0.46 and 0.87. 10,000: 0.67 and 1.22. 20,000: 0.85 and 1.50.

What to notice: with a cache active, the ratio crosses 1.0 somewhere between 3,000 and 10,000 tokens of shared background. Past that point, the single cached agent is cheaper than paying the shared-context tax across multiple workers.

How to read the scenario and reliability rows: two named task shapes, then the reliability comparison under two different hazard-rate assumptions.

Parallel research (8 independent sub-tasks, with a prompt cache): tokens 0.87, latency 0.23. Tightly coupled refactor (density 0.8, with a prompt cache): tokens 0.90, latency 1.24.

Reliability under a robust model (error rates growing 4 to 5 times more slowly): density 0 gives split 0.80 against single 0.70. Density 0.5 gives 0.60 against 0.70. Density 1.0 gives 0.46 against 0.70.

What to notice: the same two scenario names, "parallel research" and "tightly coupled refactor", get nearly identical token ratios (0.87, 0.90). But their latency ratios are wildly different (0.23, 1.24). Tokens and latency are answering different questions. A design that saves one can lose the other.

{{svg:shapes|The same eight sub-tasks as independent research and as a dependent chain.}} {{svg:results|Latency ratio by dependency density, token ratio by task size with and without a prompt cache, and reliability under two sets of hazard rates.}}