CSE Roadmap

Common tier · about 4 hours · 11 parts

Part 6 of 11 · 1 min read

How to set rank, alpha and the rest in practice

  • Start with rank 8 to 16 on all attention matrices (add MLP matrices if quality is short) and sweep rank on a validation set.
  • Initialise B to zero and A small random, so training starts at the base model.
  • Tune alpha and the learning rate together; keep alpha : r within a sensible band and watch for divergence.
  • Merge for one model. Keep adapters separate for multi-tenant or multi-task serving.
  • Version adapters with the base model, since an adapter is only valid for the exact base weights it was trained on.
  • Use 4-bit bases (QLoRA) when memory is the limit, and compare against a 16-bit run on your own eval.
  • Smaller quantisation blocks cost extra bits; measure error against memory.
  • Hold out data from a later period to check the adapter does not overfit.
  • Run a regression eval on tasks the adapter should not change.
  • Recompute any parameter count you copy (tables sometimes label the matrices wrongly).