Part 6 of 8 · 3 min read
Reasoning models and model names
Models that think before answering, what they cost, and how to read what a model name tells you.
Reasoning model
A model further trained, usually with reinforcement learning on tasks whose answers can be checked (maths, code), to produce a long chain of intermediate "thinking" tokens before the final answer.
The reward is simple: correct final answer, good. Over training the model learns to try approaches, check itself and backtrack. What it means for you:
- Better at multi-step maths, code and planning.
- Slower and costlier: thinking tokens are generated and usually billed as output tokens, even if you never see them. (Check your provider's billing page.)
- Not always needed. A reasoning model for a simple classification is a waste. Many providers let you set how much it thinks.
Reading a model name
Model names often carry these stages, though providers use their own words. A base model is stage 1 only. Instruct or chat usually means stages 2 and 3. A name with reasoning or thinking usually means stage 4. When you fine-tune your own model (Level 13), you almost always start from an instruct model and add a small stage of SFT, rarely from a base model.
For the curious Why post-training is cheap but matters so much
Pretraining uses trillions of tokens and most of the compute. Post-training uses far less data, often thousands to millions of examples, but it decides the format, tone and safety of the model. This is why the same base model can sit behind very different products, and why small fine-tunes can change behaviour a lot while adding little knowledge.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Choosing base, instruct or reasoning | You pick the stage that fits: instruct for most tasks, reasoning only for hard multi-step ones, base for custom fine-tuning. |
| Using the right chat template | With open models, you apply the tokenizer's template instead of building the string yourself. |
| Escaping user text | You never put raw user input where special tokens or structure could be forged. |
| Cost control for reasoning | You set a thinking budget or choose a smaller model, and watch output-token use. |
| Checking for agreeable answers | You test whether the model caves to pushback, since preference tuning can make it too agreeable. |
Draft: to be checked against the current documentation of the providers the course uses before this page goes live.
Interview angle
Answer in your head first, then open it.
What is the difference between a base model and a chat model?
A base model predicts the next token; a chat model has also been trained on instruction examples and human preferences, so it answers and stops.
What does SFT change, and what does it not?
It teaches format and behaviour; it is inefficient for adding lots of new knowledge.
RLHF versus DPO?
RLHF trains a reward model and optimises against it with reinforcement learning; DPO optimises directly on preference pairs and is simpler and more stable.
Why mask the prompt in the loss?
So the model learns to produce replies, not user messages.
What does a reasoning model cost you?
Latency and thinking tokens, in return for accuracy on hard problems.