CSE Roadmap

Common tier · about 3 hours · 12 parts

Part 9 of 12 · 2 min read

Mistakes that break streams in production

Common mistakes, and what the example does not do.

Mistakes that break streams in production

  • Streaming tokens only, leaving tool waits silent.
  • No event ids, so reconnect means restart or duplicates.
  • Events held in server memory (lost on deploy or on another instance).
  • One network chunk assumed to be one event in the parser.
  • Proxy buffering left on, or compression that buffers.
  • No heartbeats behind an idle-timeout proxy.
  • Dropping approval events under backpressure.
  • WebSockets chosen by default, adding session affinity for no need.
  • Leaking sensitive intermediate content in progress events.

What the example does not do

About this example What the example does not do
  • No real server, browser or proxy; buffering and timeouts are modelled, not observed.
  • Infrastructure claims (session affinity, horizontal scaling, memory) come from general knowledge; they were not measured.
  • The parser covers the common fields only (no BOM handling or retry use).
  • Event store is a list; durability, ordering across writers and retention are not modelled.
  • Backpressure is a toy queue, with no real socket write timing.
  • Optional real experiment: put your agent behind your real gateway, stream a run with a 90-second tool call, and record the first-byte time, the disconnect time with and without heartbeats, and the duplicate count on reconnect with and without Last-Event-ID.

Check yourself

A browser drops the stream after 20 of 50 events and reconnects. How do you avoid repeated text?

Show answer

Give every event an id and have the client send Last-Event-ID, so the server replays only what was missed. A blind reconnect received 70 events, 20 of them duplicates; Last-Event-ID received exactly 50.

For the curious A buffering proxy can hide the whole stream until it ends

With 40-byte events at 20 a second, a proxy that buffers 4 KiB delayed the first byte to 5.15 s, against 0.05 s unbuffered. A proxy that closes idle connections after 60 s closed a stream at 64 s while a tool ran silently, and a 15 s comment heartbeat kept it open. Those timeouts and sizes are the assumptions, so test through your real gateway.

What you will use daily at work

Practice What it looks like
Default to SSE plus saved context WebSockets only for real two-way needs.
Send typed events with ids Back them with a durable per-run event store.
Send heartbeats under the idle timeout A comment line costs almost nothing.
Never drop approval or final events Drop old token deltas first when a client is slow.