CSE Roadmap

Specialist tier · about 5 hours · 12 parts

Part 10 of 12 · 1 min read

Quick notes

  • "Open" is a spectrum: open weights is not open data, and licences differ on commercial use, size limits and whether outputs may train other models. Read the current text.
  • Seven axes for self-host versus API: data privacy, lineage, performance, functionality, cost, control and on-device. APIs give the strongest models and features; self-hosting gives control and a frozen version.
  • Will it fit, and how fast? Weights are parameters times bytes per weight, plus headroom and the KV cache; decode speed at batch 1 is bounded by bandwidth over weight bytes.
  • Self-host economics are about utilisation. A busy, batched GPU beats the API price by a wide margin; an idle one loses, and engineering time can cost more than the API bill it saves.
  • Cascades and routers save money only with a good signal and a capable small model, and switching models is a statistical decision that needs thousands of paired items to show a small gap.