CSE Roadmap

Specialist tier · about 3 hours · 12 parts

Part 6 of 12 · 2 min read

What the simulation measured

Pipeline latency, the endpointing sweep, barge-in timing and tool-silence results.

How to read the pipeline table: all three rows face the same assumed stage latencies except for the one changed per row; read it as what one change buys over the batch baseline.

Pipeline (assumed stage latencies) p50 gap p95 gap
Batch (wait for full reply, then synthesise) 2,399 ms 2,885 ms
Streaming 1,764 ms 2,268 ms
Streaming, faster model 1,564 ms 1,861 ms

What to notice: streaming alone saves about 635 ms at the median, more than switching to a faster model on top of it saves again (200 ms). The single architectural change (stream, don't batch) matters more than the model swap.

How to read the endpoint-threshold table: each row is a different silence-threshold setting; the two right-hand columns always move in opposite directions as the threshold rises.

Endpoint threshold Median gap Utterances cut off mid-pause
300 ms 1,464 ms 57%
450 ms 1,614 ms 33%
600 ms 1,764 ms 18%
800 ms 1,964 ms 8%
1,000 ms 2,164 ms 4%

What to notice: there is no row where both columns are small. Cutting the gap in half (1,000 ms to 300 ms for the threshold) means accepting cut-ins fourteen times as often (4% to 57%). That's the trade-off any single fixed threshold has to accept.

How to read the barge-in table: the four rows are four separate measurements, not one progression — read each row against what it varies.

Barge-in (35-word reply, random interruption point) Result
Talk-over at detection 100 / 150 / 300 / 600 ms 149 / 198 / 345 / 632 ms
Words recorded as said but never heard, full-reply history 17 on average
Same, history truncated at the interruption 0
False barge-ins in 10 minutes, minimum speech 0 / 200 / 250 ms 30 / 6 / 2

What to notice: the second and third rows are the same scenario with one line of code different (truncate=True versus False). The entire 17-words-to-0 improvement comes from fixing the history, not from detecting the interruption any faster.

How to read the tool-silence table: both rows face the same simulated tool-latency distribution (median 900 ms); only whether a filler plays differs.

Tool latency (median 900 ms) p50 silence p95 silence
No filler 895 ms 2,376 ms
Filler played at once 250 ms 1,176 ms

What to notice: the filler doesn't make the tool faster — the tool still takes just as long — it only changes what the caller perceives, which is the entire point of playing one.

{{svg:budget|Where the silence comes from, and three pipelines.}} {{svg:results|Endpointing, interruption and tool-silence trade-offs.}}