Part 1 of 4 · 1 min read
Overview
Most agent failures that tuning can fix are about multi-step success: calling the right tools in the right order. A pass/fail check on the final state (tests pass, ticket resolved, form submitted) is a verifiable reward. This module explains why such training is data-hungry for long tasks, why it can silently learn to cheat the checker, and what you must build (environments, verifiers, held-out audits) before RL is worth trying. It also explains why most teams stop at prompting, retrieval and SFT.
About the example, and what is real
Real: the policy-gradient code with hand-derived gradients (checked against finite differences), the group-relative, leave-one-out, batch-baseline and REINFORCE advantages, PPO-style clipping, KL-shaped rewards, the verifiers, and the 18 tests in code/mini-rlvr. Assumed: the task (a target chain per prompt), the number of prompts, steps and tokens, learning-rate grids, and the particular weak verifier and loophole (the loophole is discoverable by chance, 20 to 35 percent of the time at the start). Not real: no language model, no reasoning text, no value network, no GPU; the numbers show mechanisms, not the behaviour of real reasoning models.