Part 8 of 12 · 2 min read
Answering prioritisation and exploration questions in an interview
How would you use extra compute to make an agent more accurate?
"Repeated sampling with a way to choose: generate N candidates and select with a verifier, ideally something deterministic like tests. In the example's simulation best-of-3 with a verifier took accuracy from 0.55 to 0.73 for 6 calls, and returns diminished after that (0.82 at 15 samples and 30 calls). Majority vote is cheaper but only helps when wrong answers are scattered; when half of them agreed it capped at 0.65." Follow-up: "What if you have no verifier?" Then I rely on self-consistency for short-answer tasks, and measure how correlated the errors are, because if wrong answers agree the vote gains little.
How does an agent decide which task to do first?
"With a rule that matches the objective: value per cost under a budget, with deadlines and dependencies as constraints. On my simulated queue it reached 99 percent of the optimal value against 58 percent for arrival order. But if an agent estimates the values itself, the estimate may be noisy: at 100 percent error, ranking by value per cost fell to 0.84, below simply taking the cheapest first at 0.88. So I calibrate the estimates against outcomes before relying on them." Follow-up: "What about dependencies?" Rank by the value unlocked, including downstream tasks, and respect the dependency order (module 1 of this level).
Explain exploration against exploitation for agents.
"If I have several prompts, tools or sub-agents and don't know which works best, always using the current favourite risks missing a better one, and trying them all equally wastes runs. A bandit policy such as epsilon-greedy or Thompson sampling balances them. In the example's simulation never exploring had regret growing linearly (32 at 100 runs), while adaptive policies grew slowly (8 at 100 runs). If the environment changes, for example a tool breaks, a policy that committed after an initial phase kept paying: 114 against 83 for UCB over 1,000 runs." Follow-up: "What do you need to use it?" A per-run success signal, enough volume (at 20 runs there is little to learn), and a reason to keep a small exploration share.
Why does majority voting fail sometimes?
"Because it assumes errors are independent. If a task has a tempting wrong answer, many samples converge on it and the vote amplifies the mistake: in the example's test the same N=15 gave 0.72 when wrong answers scattered and 0.58 when 90 percent shared one. A verifier does not have that problem, so it stayed near 0.82, but it has its own error: a verifier that approves everything gives no gain." Follow-up: "How would you detect it?" Measure agreement on a labelled set and check whether high-agreement answers are actually right (calibration).