Part 10 of 12 · 1 min read
Quick notes
- Finetuning is for form, RAG is for facts. Information failures call for retrieval; behaviour failures (format, style, rare syntax) make fine-tuning a candidate. With both, start with retrieval.
- Tune only after systematic prompt experiments on an eval set; "prompting did not work" often means three prompts were tried.
- Good reasons to tune: a narrow task, strict structured output, uncommon syntax, distilling a large model into a small one. Reasons not to: it can degrade other tasks, needs data and serving, and a new base model may overtake it.
- Tuned facts go stale while retrieval keeps up, and domain pretraining is no guarantee against a strong general model; compare on your own task.
- Do the break-even. Up-front cost, monthly upkeep and periodic re-tunes mean low volumes never pay back, and prompt caching weakens the cost argument further.