Thinking longer, voting, checking

Three ways to spend more compute

1. Think longer. The model writes intermediate steps before the answer. Those tokens are working memory: each written step can be checked against the one before, so a long problem is broken into pieces the model gets right more often. A reasoning model is trained to do this by itself, usually in a hidden "thinking" section, and many APIs let you set a thinking budget (a cap on those tokens).
2. Try many times. Ask the same question several times (with some randomness) and combine the answers: by majority vote (called self-consistency) or by picking the one a checker likes (best of N).
3. Both. Several attempts, each thinking.

Thinking longer

Take problems that need 1 to 8 steps. The chance that one attempt is right, by how many thinking tokens are allowed:

Each 60 tokens of budget pays for one step to be written out. A budget smaller than the steps needed leaves the later steps to the head (the diagonal "staircase" in the table). A budget larger than the steps needed is simply not used, because the model stops when it is done. Averaged over a mix of problems:

Two features to remember. Returns diminish: the first 240 tokens add 29 points; the next 240 add 10. And the cost is concentrated on hard problems: the average at budget 480 is only 254 tokens because easy problems stop early, but a hard problem uses all 490.

Trying many times and voting

With no thinking tokens, ask n times and take the most common answer:

Voting amplifies whichever answer is most common. On easy problems the right answer is the most common, so each extra attempt helps. On hard problems the attractive wrong answer is more common than the right one, so voting makes the answer more reliably wrong: 12% falls to 0%. Overall the curve flattens at 50%, however many attempts you add.

This depends on a property of real models: do wrong answers scatter or cluster? If they all differ, the right answer wins by plurality even when it is rare. In the toy, setting the attractive share to zero lets voting reach 57% at 9 attempts and 70% at 33. Real models often make the same mistake again (the same misreading, the same missing fact), which is why voting alone is not a cure.

Picking with a checker

If something can tell right from wrong, take the first attempt it accepts:

(No thinking tokens, and a rejected answer is replaced by another attempt, falling back to the majority if none is accepted.)

• A perfect checker climbs steeply: n attempts is right if any one is right.
• A good but imperfect checker plateaus (here near 68%), set by how often it lets a wrong answer through. The false-accept rate, not the number of attempts, is the limit.
• A poor checker gives a small gain that stops near 50%.
• Checking stops early, so it costs fewer tokens than the full n attempts (about 24 tokens at 17 attempts, not 170).

This is module 4's lesson again: the checker is only worth what its outside signal is worth. A model grading another model's answers has the same trouble as a model critiquing itself.