Part 4 of 6 · 6 min read
Interview questions and limits
Covers: Interview questions, with the follow-ups; Common mistakes; What the example does not do.
Interview questions, with the follow-ups
Two agents update the same shared state at the same time. What can go wrong and how do you prevent it?
"The classic failure is a lost update: both read the same value, each computes a new one, and the second write silently overwrites the first. In my simulation with six agents, last-writer-wins kept 6 of 30 items and corrupted a running total. I prevent it by turning writes into operations the store can merge (append, add), by optimistic concurrency with a version check, or by giving each field one owner. Optimistic concurrency is correct but each conflict costs a model call: model calls went from 30 to 113 in my test." Follow-up: "Why is that worse for agents than for a database?" The window between read and write is an LLM call, so conflicts are likelier, and a retry repeats an expensive call. Follow-up: "When can't you merge?" Decisions, such as two agents claiming the same task: you need compare-and-set, a priority rule, or a person.
How do you make agent-to-agent messaging reliable?
"I treat the channel as unreliable: messages can be lost, duplicated and reordered. I use at-least-once delivery with bounded retries, give every message an id and make the receiver idempotent, and add sequence numbers if order matters. At-most-once lost 20 of 100 messages in my test, plain at-least-once repeated 35 effects, and keys plus sequence numbers gave zero missing, zero repeated and zero out of order, with the same 153 messages sent." Follow-up: "Is exactly-once delivery possible?" Not as a channel guarantee; what you get is exactly-once effect from at-least-once delivery plus idempotent processing.
Shared blackboard or message passing?
"It depends on who owns the data. A blackboard is simple and good for append-only contributions that commute, like findings; it needs versions or merge rules for anything else, and every agent can see everything, which costs tokens and raises exposure. Message passing gives clear ownership, isolation and replay, at the price of protocol design: ids, retries, ordering. I use scoped views either way: in my test, showing all twelve agents the whole state cost 260,400 input tokens against 83,280 for a scoped view." Follow-up: "What about security?" A shared blackboard lets one compromised agent write to everyone's context, which is a prompt-injection path; scoped views and validated message schemas limit it.
What is idempotency and why do agents need it?
"An operation is idempotent if doing it twice has the same effect as doing it once. Agents retry (a tool call times out, a message is redelivered, a run is resumed from a checkpoint), so any side effect, such as sending an email or creating a record, must carry an idempotency key that the receiver checks." Follow-up: "Where does the key come from?" The caller generates it from the logical operation (task id plus step), not from the attempt, so a retry reuses it.
Common mistakes
- Replacing the whole state on every write.
- Retrying without idempotency.
- Optimistic retries with no back-off or limit.
- Merging decisions as if they were counters.
- Broadcasting the full state into every prompt.
- Assuming messages arrive in order.
- Losing the trace: no ids or versions in logs, so a lost update cannot be found.
- A shared state that every agent may write, including agents that read untrusted content.
What the example does not do
- No real reasoning. An agent's update is deterministic; real agents may produce inconsistent updates even without concurrency.
- A single store. No replication, partitions or clocks; those are database topics.
- No back-off. Retries are immediate, which exaggerates contention.
- A fixed, simple channel. Real channels add authentication, backpressure and security.
- Optional real-system experiment (I did not run this): run three real agents that append to a shared file or a Redis stream with and without versions, and count lost entries.
Check yourself
Two agents read the same list, each adds an item, and each writes the list back. One item is missing. What happened, and what are two fixes? (A lost update: the second write replaced the first. Write only if the version is unchanged and retry, or send the operation "append" instead of the new list.)
For the curious Retries cost tokens, not milliseconds
Optimistic concurrency lost no items but needed 83 retries, and every retry re-runs the agent reasoning, so model calls rose from 30 to 113. Merging by operation lost nothing with no retries, but only works when the order does not matter. A contested claim, where several agents want the same task, cannot be merged and needs a visible conflict or a person.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| One writer per field | Other agents send messages to the owner. |
| An idempotency key on every message | At-least-once delivery plus an idempotent receiver gives the effect of exactly once. |
| Show agents a scoped view | Twelve agents each shown everything used 260,400 input tokens; a scoped view used 83,280. |
Draft: confirm what your message queue or database guarantees about ordering and duplicates.
What you must be able to say (one page)
- Agents coordinate through shared state (a store everyone reads and writes, often called a blackboard) or through messages (point to point or through a bus).
- Shared state fails by lost update. Two agents read the same value, each computes a new value, and the second write erases the first. It is not a rare event: with six agents each adding five items, last-writer-wins kept 5.9 of 30 items and corrupted the running total.
- Three fixes, with three different costs: 1. Optimistic concurrency (write only if the version is unchanged): lost 0 items, but needed 83 retries, and every retry re-runs an agent's reasoning, so model calls went from 30 to 113 (from 60 to 387 with twelve agents). 2. Merge by operation (send "append" or "add", not a new value): lost 0, no retries, 30 calls; but it only works for operations that commute (order does not matter). 3. One writer per field, others send messages to the owner (the actor style): removes the race by design.
- A contested decision cannot be merged. If five agents all claim the same task, an overwrite and a merge both pick a winner silently; only a compare-and-set, a priority rule or a person makes the conflict visible (the optimistic policy detected all 5).
- Messaging fails by loss, repetition and reordering. On a channel that drops 20 percent of messages: at-most-once lost 20 of 100; at-least-once lost none but repeated 35 effects (and sent 153 messages for 100); adding an idempotency key removed the repeats at no extra traffic; adding sequence numbers removed the reordering (27 inversions down to 0).
- "Exactly once" in practice is at-least-once delivery plus an idempotent receiver.
- What an agent is shown matters as much as where state lives. Showing every agent the whole shared state cost 260,400 input tokens for twelve agents over five rounds; a scoped view cost 83,280 and pointers 78,240 (an assumed 120 tokens per entry).