Part 6 of 12 · 2 min read
What the simulation measured
Pipeline latency, the endpointing sweep, barge-in timing and tool-silence results.
How to read the pipeline table: all three rows face the same assumed stage latencies except for the one changed per row; read it as what one change buys over the batch baseline.
| Pipeline (assumed stage latencies) | p50 gap | p95 gap |
|---|---|---|
| Batch (wait for full reply, then synthesise) | 2,399 ms | 2,885 ms |
| Streaming | 1,764 ms | 2,268 ms |
| Streaming, faster model | 1,564 ms | 1,861 ms |
What to notice: streaming alone saves about 635 ms at the median, more than switching to a faster model on top of it saves again (200 ms). The single architectural change (stream, don't batch) matters more than the model swap.
How to read the endpoint-threshold table: each row is a different silence-threshold setting; the two right-hand columns always move in opposite directions as the threshold rises.
| Endpoint threshold | Median gap | Utterances cut off mid-pause |
|---|---|---|
| 300 ms | 1,464 ms | 57% |
| 450 ms | 1,614 ms | 33% |
| 600 ms | 1,764 ms | 18% |
| 800 ms | 1,964 ms | 8% |
| 1,000 ms | 2,164 ms | 4% |
What to notice: there is no row where both columns are small. Cutting the gap in half (1,000 ms to 300 ms for the threshold) means accepting cut-ins fourteen times as often (4% to 57%). That's the trade-off any single fixed threshold has to accept.
How to read the barge-in table: the four rows are four separate measurements, not one progression — read each row against what it varies.
| Barge-in (35-word reply, random interruption point) | Result |
|---|---|
| Talk-over at detection 100 / 150 / 300 / 600 ms | 149 / 198 / 345 / 632 ms |
| Words recorded as said but never heard, full-reply history | 17 on average |
| Same, history truncated at the interruption | 0 |
| False barge-ins in 10 minutes, minimum speech 0 / 200 / 250 ms | 30 / 6 / 2 |
What to notice: the second and third rows are the same scenario with one line of code different (truncate=True versus False). The entire 17-words-to-0 improvement comes from fixing the history, not from detecting the interruption any faster.
How to read the tool-silence table: both rows face the same simulated tool-latency distribution (median 900 ms); only whether a filler plays differs.
| Tool latency (median 900 ms) | p50 silence | p95 silence |
|---|---|---|
| No filler | 895 ms | 2,376 ms |
| Filler played at once | 250 ms | 1,176 ms |
What to notice: the filler doesn't make the tool faster — the tool still takes just as long — it only changes what the caller perceives, which is the entire point of playing one.
{{svg:budget|Where the silence comes from, and three pipelines.}} {{svg:results|Endpointing, interruption and tool-silence trade-offs.}}