CSE Roadmap

Specialist tier · about 3 hours · 12 parts

Part 3 of 12 · 2 min read

Endpointing trades speed for cutting people off

A shorter silence threshold answers faster and interrupts more often — there is no setting that avoids both.

Think of a radio host who jumps in the instant a guest pauses for breath, versus one who waits so long that every answer feels like dead air. Neither is simply "better." They trade off against each other.

In plain words: lowering the endpointing threshold makes the agent feel faster to respond. But it also makes the agent start talking during an ordinary mid-sentence pause more often. There's no threshold that is simply correct.

A worked example. Sweeping the threshold from 300 ms to 1,000 ms: at 300 ms the agent cuts people off mid-pause in 57 percent of utterances, with a 1,464 ms median gap. At 1,000 ms it cuts people off only 4 percent of the time, but the median gap rises to 2,164 ms. Every 100-200 ms of threshold trades a few percentage points of cut-ins for a few hundred milliseconds of gap.

The key lines.

# from code/mini-voice/latency.py
def cut_in_rate(endpoint_ms, n=20000, seed=2, pauses_per_utterance=1.5):
    """Share of utterances where the agent starts talking during a mid-sentence pause longer than the endpoint threshold."""
    rng = random.Random(seed)
    cut = 0
    for _ in range(n):
        k = max(0, round(rng.gauss(pauses_per_utterance, 1)))
        if any(lognorm(rng, 300, 0.6) > endpoint_ms for _ in range(k)):
            cut += 1

Read it as: the cut-in rate is simulated from a pause-length distribution, not from a fixed rule. Real speakers pause for unpredictable amounts of time. That's exactly why a single fixed threshold can never get both numbers to zero at once.

VAD (voice activity detection)

VAD is detecting whether audio contains speech at all — the lower-level signal that endpointing's silence-timer builds on.

Example: the 300 ms median pause length assumed in cut_in_rate above is itself the output a VAD stage would have to supply correctly for the threshold sweep to mean anything on real audio.