Part 9 of 12 · 2 min read
Mistakes that break streams in production
Common mistakes, and what the example does not do.
Mistakes that break streams in production
- Streaming tokens only, leaving tool waits silent.
- No event ids, so reconnect means restart or duplicates.
- Events held in server memory (lost on deploy or on another instance).
- One network chunk assumed to be one event in the parser.
- Proxy buffering left on, or compression that buffers.
- No heartbeats behind an idle-timeout proxy.
- Dropping approval events under backpressure.
- WebSockets chosen by default, adding session affinity for no need.
- Leaking sensitive intermediate content in progress events.
What the example does not do
About this example What the example does not do
- No real server, browser or proxy; buffering and timeouts are modelled, not observed.
- Infrastructure claims (session affinity, horizontal scaling, memory) come from general knowledge; they were not measured.
- The parser covers the common fields only (no BOM handling or
retryuse). - Event store is a list; durability, ordering across writers and retention are not modelled.
- Backpressure is a toy queue, with no real socket write timing.
- Optional real experiment: put your agent behind your real gateway, stream a run with a 90-second tool call, and record the first-byte time, the disconnect time with and without heartbeats, and the duplicate count on reconnect with and without Last-Event-ID.
Check yourself
A browser drops the stream after 20 of 50 events and reconnects. How do you avoid repeated text?
Show answer
Give every event an id and have the client send Last-Event-ID, so the server replays only what was missed. A blind reconnect received 70 events, 20 of them duplicates; Last-Event-ID received exactly 50.
For the curious A buffering proxy can hide the whole stream until it ends
With 40-byte events at 20 a second, a proxy that buffers 4 KiB delayed the first byte to 5.15 s, against 0.05 s unbuffered. A proxy that closes idle connections after 60 s closed a stream at 64 s while a tool ran silently, and a 15 s comment heartbeat kept it open. Those timeouts and sizes are the assumptions, so test through your real gateway.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Default to SSE plus saved context | WebSockets only for real two-way needs. |
| Send typed events with ids | Back them with a durable per-run event store. |
| Send heartbeats under the idle timeout | A comment line costs almost nothing. |
| Never drop approval or final events | Drop old token deltas first when a client is slow. |