CSE Roadmap

Common tier · about 3 hours · 12 parts

Part 8 of 12 · 1 min read

Answering streaming questions in an interview

SSE or WebSockets for an agent chat UI?

"I start with SSE plus context saved between interactions: it is plain HTTP, load-balances without session affinity and supports an approval that waits hours, which feels the same to the user. I use WebSockets only for real-time bidirectional needs like collaborative editing." Follow-up: "How does the user interrupt?" A normal cancel request to the job API (module 1); the stream then emits a cancelled event.

The connection drops mid-run. What happens?

"The run continues on the worker; events are in a durable store with ids. The client reconnects with Last-Event-ID and the server replays what it missed. In the example's test, blind reconnect produced 20 duplicate events. Last-Event-ID produced none and lost none." Follow-up: "Where do events live?" In the job's event log or journal, in a database or stream, not in the web server's memory.

Streaming works locally but arrives in one lump in production. Why?

"Something in the path buffers the response: a reverse proxy, CDN or gzip. In the example's simulation a 4 KiB buffer delayed the first byte from 0.05 s to over 5 s. I disable buffering for the route, flush per event, and test through the real gateway." Follow-up: "And it dies after a minute?" Idle timeout: add heartbeats shorter than the limit.

What do you do about a slow client?

"Bound the buffer and choose a policy per event type: drop old token deltas, never drop approvals or the final event, or disconnect and let the client resume. Unbounded buffers turn one slow phone into server memory growth." Follow-up: "What do you stream when the model is thinking?" Progress events that are safe to show, not hidden reasoning you would not want exposed.