Part 3 of 12 · 2 min read
Resolution rate moves margin more than token price does
Engineers reach for the model-price lever first; the sensitivity numbers say to reach for the resolution-rate lever instead.
Think of a restaurant that assumes raising menu prices is the way to fix thin margins. The real problem is that a third of orders are sent back to the kitchen and remade for free.
In plain words: when several inputs can each move the margin, the honest way to find the biggest lever is to move each one by the same percentage and compare — and for this support task, resolution rate wins by a wide margin over model price.
A worked example. Moving each input by 20 percent against a $0.60 task price and a 63 percent base margin: a 20 percent drop in agent resolution rate costs 27.5 points of margin — by far the largest single effect. A 20 percent rise in the failure credit costs 4.7 points; a 20 percent rise in model price costs 2.2 points; 20 percent more tokens per task costs another 2.2 points. Oddly, a 20 percent rise in the escalation rate actually raises margin by 13.5 points, because an escalation is cheaper for the vendor than a failure — but it lowers the value the customer is getting, so it's not something to optimize toward.
The key lines.
# from code/mini-pricing/economics.py
def sensitivity(a, cs, factor=0.2):
"""Change in blended margin under task pricing when one input moves by +factor."""
base = evaluate("task", scaled("task", 1.0), cs, a)["margin"]
moves = {"model price": dict(price_in=a.price_in * (1 + factor), price_out=a.price_out * (1 + factor)),
"cache hit share": dict(cache_share=max(0, a.cache_share * (1 - factor))),
"agent resolution rate": dict(p_agent=a.p_agent * (1 - factor)),
Read it as: every input is moved by the same factor and compared against the same base margin. That is what makes the resulting list of point-changes a fair comparison of levers. It is not an arbitrary set of what-ifs.
Gross margin
Gross margin is (revenue minus the direct cost of serving) divided by revenue — the fraction of what a customer pays that is actually left after covering the cost to deliver.
Example: the "+13.5 points" escalation-rate result above is a gross-margin change — it says nothing about whether customers are happier, which the margin number alone can never tell you.