CSE Roadmap

Specialist tier · about 3 hours · 12 parts

Part 10 of 12 · 1 min read

Quick notes

  • A voice agent is a real-time pipeline: audio in, endpointing, speech-to-text, the model and tools, text-to-speech, audio out. Latency is the product, so think in a turn-gap budget where every stage counts.
  • Endpointing is a trade-off, not a setting to minimise: a short wait feels fast but cuts people off mid-pause.
  • Interruptions are part of the design: stop playback, flush the audio buffer, cancel generation, and truncate the history to what the user actually heard. Require a minimum speech duration so a cough does not count.
  • Write for the ear: short sentences, two sentences then "want the rest?", numbers in small groups, critical details confirmed back. Play a short filler when a tool will take longer than about 700 ms.
  • Production concerns: phone-grade audio, streaming everywhere, no half-heard commands executed, confirmation before irreversible actions, disclosure and consent to record, and cost per minute. Evaluate on recorded calls.