CSE Roadmap

Core tier · about 3 hours · 12 parts

Part 7 of 12 · 1 min read

How to build it so it holds up

  • Put timeouts on every call and a deadline on every run; keep inner timeouts shorter than outer ones.
  • Retry only transient errors, at one layer, with backoff, jitter and a budget; honour Retry-After; only retry idempotent or keyed operations.
  • Use a circuit breaker per dependency (per provider, per tool) with a probe; alert when it opens.
  • Define a fallback chain and test it: secondary provider or model, cached answer, degraded tools, human handoff; log which path served each request.
  • Check fallback permissions (data residency, regulation, quality floor) before using them (Level 10 module 10).
  • Rate-limit on the client with a shared token bucket; queue background work, shed interactive work.
  • Isolate with bulkheads: per-tool and per-tenant pools or concurrency limits (module 7).
  • Degrade honestly: tell users when an answer is reduced; do not silently return stale or weaker output for critical tasks.
  • Run failure drills: inject outages, slowness and 429s in staging and watch the dashboards (module 11).
  • Alert on symptoms (error rate, p95, breaker state, fallback rate) not just causes.