Part 4 of 7 · 2 min read
Changing the embedding model
Why a half-re-embedded index fails, and how to switch safely.
Think of switching map systems: coordinates from one map mean nothing on another. Vectors from different embedding models live on different maps, so you cannot mix them in one index.
A vector from one model cannot be compared with a vector from another, even if both have the same number of dimensions. Their spaces are unrelated. If you switch the query side to a new model while the index still holds old vectors, the old chunks score essentially at random.
The example re-embeds a growing fraction of a 48-chunk index with a new model while queries already use it:
| Re-embedded | Questions answered right |
|---|---|
| 0% | 6% |
| 25% | 25% |
| 50% | 48% |
| 75% | 73% |
| 100% | 98% |
Accuracy tracks the fraction almost one-for-one: a half-migrated index is a half-broken index. (The 98% at the end, against 100% for the old model on its own, is a reminder that a "better" model is not better until you have measured it on your data. See module 7.)
Safe pattern, often called blue-green:
- Build a second index with the new model next to the old one. Keep serving from the old one.
- When it is complete, run your golden questions (module 7) on both. Switch only if the new one is at least as good.
- Switch the query side and the index together (one configuration change), keep the old index for a while, then delete it.
- Stamp every index with a configuration id (model, chunker version, text normaliser) and refuse to search when the query side's id differs. In the example, a "strict" search raises an error on mixed vectors, which is better than quietly returning nonsense.
Anything that changes vectors counts: the model, its version, the tokenizer, how you clean text before embedding, the chunk boundaries, and the prefix some models expect on queries.