Part 5 of 7 · 3 min read
Partial JSON and agent streams
Show half-written JSON for display only, stream the steps of an agent, and keep a checklist.
Showing partial JSON
Sometimes the output is JSON and you want to show fields as they appear. You cannot call json.loads on half a document. preview_partial_json closes whatever is open (a string, a list, an object) and drops a dangling key. Real results:
| Received so far | Preview |
|---|---|
{"city": "Pu |
{"city": "Pu"} |
{"city": "Pune", "tags": ["a", "b |
{"city": "Pune", "tags": ["a", "b"]} |
{"city": "Pune", "temp": |
{"city": "Pune"} |
{"a": "x", "b" |
{"a": "x"} |
{"a": tru |
nothing (None), because tru is not a value yet |
For display only
A preview is a guess about a value that is still being written: "Pu" is not the final city. Never save it, validate it or act on it. Decide only when the stream has finished and the full text validates (module 3). Also note that the function gives up (None) on a half-typed true, null or number such as 1e. That is acceptable for a display, and it is one more reason not to trust a preview.
Streaming in an agent
An agent loop has two kinds of output to stream: the model's text, and the steps (calling a tool, tool finished, writing the answer). Streaming the steps is often more valuable than streaming words, because it tells the user what is happening during the long waits. A common design is one event stream with a type field (text, tool_start, tool_end, done, error), the same shape as the model's own stream. Level 11, module 4 covers sending such a stream to clients at scale.
When the model's reply contains tool calls, providers stream the arguments in pieces too. Assemble the whole argument string, and parse it only after the call is complete (module 4).
Checklist
- Show the first text quickly, and keep a spinner for the wait before it.
- Handle cancel, timeout and a dropped connection as normal cases, not surprises.
- Log usage from the final event (module 6). If there is none, estimate and mark it as an estimate.
- Do not act on partial output. Act on finished, validated output.
- Test with a fake stream that breaks at every position. The example's tests use
fail_afterfor this.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Official SDK streaming helper | You iterate over the SDK's stream object and let it do the SSE parsing. |
| Stop button | The interface cancels the request, and the server closes the upstream call too, so it stops paying. |
| Stall timeout | If no event arrives for a set time, the call is treated as failed and retried or reported. |
| Step events | The agent sends "searching", "reading file" and "writing" events, not only words. |
Draft: confirm event names, usage fields and cancellation behaviour against your provider's current documentation before relying on them.
Interview angle
Answer in your head first, then open it.
What does streaming improve?
The time to the first visible text. Not the total time and not the cost.
How do you cancel a streaming request?
Close the stream or connection. Tokens already generated are still billed.
How do you parse SSE?
Buffer the incoming chunks, split on blank lines, parse each data: payload, and keep the remainder for the next chunk.
What goes wrong with chunked input?
Events and line endings can be split anywhere, so test with every chunk size.
Can you show half-written JSON?
Only as a preview with open structures closed. Never parse it for decisions until the stream is complete.
What if the connection drops halfway through a tool call?
Discard the partial call and retry. Never run a tool from incomplete arguments.