Part 3 of 12 · 2 min read
A verifier is a filter, and a filter is only as good as what it lets through
A verifier that approves a correct answer at 95 percent is doing its job; what breaks best-of-N is how often it also approves a wrong one.
Think of airport security screening bags. A scanner that flags almost every real threat is doing its job well. But if it also waves through a decent share of actual threats by mistake, the checkpoint stops meaning much.
In plain words: a verifier is something that judges a candidate answer, false approval is the verifier accepting a wrong answer, and an attractor is a wrong answer that many samples converge on -- a verifier's false-approval rate is the ceiling on how much best-of-N can help, no matter how many candidates you draw.
A worked example. With a verifier that never approves a wrong answer, best-of-5 reaches 0.83; raise its false-approval rate to 30 percent and best-of-5 drops to 0.69; at 100 percent (it approves everything) best-of-5 falls to 0.54 -- no better than drawing one sample and trusting it, because an always-approving verifier gives the picker no real information to act on.
The key lines. From code/mini-explore/sampling.py:
# from code/mini-explore/sampling.py
if verifier:
acc = [c for c in cands if r.random() < (p.tpr if c == 0 else p.fpr)]
pick = r.choice(acc) if acc else r.choice(cands)
Read it as: each candidate c passes the filter with probability p.tpr if it is actually correct (c == 0), or p.fpr if it is wrong. The final pick is a random choice among whatever survives acc. When nothing survives, the code falls back to picking blindly among all candidates. That fallback line is itself a quiet admission. A verifier that rejects everything is exactly as unhelpful as one that approves everything.
Verifier, false approval, attractor
A verifier is something that judges a candidate answer -- tests, a checker, a reward model, a person. False approval is the verifier accepting a wrong answer. An attractor is a wrong answer that many samples converge on, so a vote follows it.
Example: best-of-5's accuracy sliding from 0.83 to 0.54 as the false-approval rate rises from 0 to 100 percent is the false-approval rate acting as a direct multiplier on how much the whole best-of-N scheme is worth.