CSE Roadmap

Specialist tier · about 5 hours · 12 parts

Part 7 of 12 · 1 min read

How to build it so it holds up

  • Write down the seven axes and which ones are hard constraints (privacy, region, on-device) before comparing models.
  • Read each licence for commercial use, user-count limits and output-use restrictions.
  • Size memory as weights plus cache, then check decode speed with a bandwidth bound before renting hardware.
  • Compute cost at realistic utilisation, not full load, and include engineering and on-call time.
  • Measure the small model on your own traffic before designing a router; the gain depends on it.
  • Make routers and confidence checks cheap and measure their own accuracy.
  • Keep an escape hatch: log escalations and sample the cheap path for review.
  • Use a paired comparison on a fixed evaluation set and plan the item count for the gap you care about.
  • Pin versions of open weights and record them with every evaluation.