Part 1 of 4 · 1 min read
Overview
Agents make many calls per task, and most of them (classification, extraction, format repair, tool-argument filling) do not need the strongest model. Sending those to a small or open model, and keeping the large one for planning and hard steps, can cut cost sharply; the risk is silent quality loss, so the routing decision and the model switch both need measured evidence.
About the example, and what is real
Real: the memory and bandwidth arithmetic, the cost-per-token and break-even formulas, a routing and cascade simulation, a sample-size and power calculation with a simulation check, and 24 tests in code/mini-small. Assumed: the device list (phone 8 GB and 50 GB/s, laptop 32 GB and 100 GB/s, 24 GB GPU at 1 TB/s, 80 GB GPU at 2 TB/s, of which only the last bandwidth is a published figure), the $2.50 per hour GPU price, the $6 per million API price, engineering cost, small and large model skills and costs, the Beta(2,2) difficulty distribution, and the confidence-signal model. Not real: no real model served, no real traffic, no trained router, no hosted-API measurements; the 8B example uses a multi-head cache layout, which overstates the cache for models with grouped heads.