Part 1 of 6 · 1 min read
Overview
In this module you will
- Compute the cost of any call from its token usage, including cached input.
- Keep a ledger that answers "what did this run, this user, this model cost?" and survives a restart.
- Stop a runaway agent with a budget guard that checks before each call, using a safe estimate.
- Lay out a prompt so a provider's prefix cache can save most of the input cost, and prove it with a simulation.
An agent re-sends its whole history on every step (module 1), so the cost per step grows through a run. A loop that gets stuck can spend real money in minutes. Logging every request's cost is only the start. You also want a cap, and you want to know which part of the bill you can shrink.
About the numbers
All prices here are placeholders (3.00 dollars per million input tokens for the "large" model, and so on). Real prices, cache discounts, minimum sizes and expiry times differ by provider and change often, so read your provider's current price page. The cache is a simulation of the general mechanism. No real provider or network was used.