Part 7 of 12 · 1 min read
How to build it so it holds up
- Write the claim in one sentence before reading the method.
- Find the test size, the seeds and the interval; if absent, treat the gain as unproven.
- Check the baseline's tuning budget and prompts against the proposed method's.
- Ask what was selected on the test set and who chose the benchmarks and when.
- Look for ablations, slices and a contamination check.
- Compare effect size with what chance produces at that test size (this module's tables).
- Reproduce the headline on your own data with a fixed budget before adopting a method.
- Keep a one-page note per paper (the template) and record what you tested.
- Report your own results with an interval, seeds and the tuning budget you gave each side.