CSE Roadmap

Core tier · about 3 hours · 12 parts

Part 7 of 12 · 1 min read

How to build it so it holds up

  • Track cost per resolved task with traces (Level 9), split by step, model and tenant.
  • Set a quality floor first, then minimise cost above it.
  • Cap every run (dollars, steps, wall time) and define what happens at the cap.
  • Cache the stable prefix: system prompt and tools first, volatile data last; measure hit rate.
  • Use exact caches for deterministic lookups; treat semantic caches as risky and keep thresholds high, scoped per tenant and excluded for sensitive queries.
  • Prefer cascades over blind routers where you can verify answers cheaply; evaluate the verifier.
  • Compress context: summarise, truncate tool output, drop stale steps (Level 7).
  • Batch non-urgent work and keep interactive traffic separate.
  • Budget latency too: TTFT, time to first progress event (module 4), total.
  • Re-evaluate routing when models or prices change; keep the policy in config, not code.