Part 8 of 12 · 2 min read
Answering orchestration questions in an interview
What is the difference between an orchestrator-workers design and a handoff design?
"In orchestrator-workers a central agent decomposes the task, dispatches parts to workers and combines the results. There is one coordinator that sees everything, and it is a single point of failure. In a handoff design, control moves peer to peer: the agent that has the task decides who gets it next and passes the context along. There is no central bottleneck, but control can bounce, which is why it needs a hop limit. On the example's test task, handoff cost fewer calls (2.3 against 3.2) and recovered misroutes, while orchestrator-workers handled tickets needing two domains more reliably." Follow-up: "Which would you pick?" Orchestrator-workers when the task must be decomposed at run time; handoff when the specialists can tell whether a request is theirs.
When is a router enough?
"When each request has one domain and the classifier is accurate. Routing is a conditional workflow: cheap, predictable and easy to test. It fails silently on requests that need two domains, because it hands the request to one specialist. On the example's task it beat the single agent with only single-domain tickets (0.76 against 0.70) and scored zero on two-domain ones." Follow-up: "How would you know?" I would log the share of requests that needed more than one specialist and watch it before choosing the shape.
When does a critic agent help, and when does it hurt?
"It helps when it catches errors the worker would not, at a cost of calls and latency. It hurts when it rejects correct work, because the revision is a fresh attempt that can be wrong. In the example's simulation, a critic that detected half the errors and rejected one in ten correct answers raised accuracy from 0.71 to 0.77 for about 1.7 more calls. A critic that detected nothing and rejected 30 percent lowered it to 0.65. I would measure the critic's detection and false-rejection rates on labelled examples before shipping it." Follow-up: "Should the critic be a different model?" Often yes, since a model reviewing its own output shares its blind spots; I would test that rather than assume it.
How do you prevent infinite loops between agents?
"A hop limit and a message limit on every conversation, plus a rule that an agent cannot hand back to the one that just sent it without new information. Add an explicit failure path, such as escalating to a human or returning a partial answer, for when the limit is reached. The limit has a cost. In the example's test, a limit of two ended some legitimate two-step handoffs, so I would set it from the longest valid path." Follow-up: "How do you find loops?" Traces: count visits per agent per task and alert on repeated visits.