CSE Roadmap

Common tier · about 4 hours · 7 parts

Part 4 of 7 · 2 min read

Changing the embedding model

Why a half-re-embedded index fails, and how to switch safely.

Think of switching map systems: coordinates from one map mean nothing on another. Vectors from different embedding models live on different maps, so you cannot mix them in one index.

A vector from one model cannot be compared with a vector from another, even if both have the same number of dimensions. Their spaces are unrelated. If you switch the query side to a new model while the index still holds old vectors, the old chunks score essentially at random.

The example re-embeds a growing fraction of a 48-chunk index with a new model while queries already use it:

Re-embedded Questions answered right
0% 6%
25% 25%
50% 48%
75% 73%
100% 98%

Accuracy tracks the fraction almost one-for-one: a half-migrated index is a half-broken index. (The 98% at the end, against 100% for the old model on its own, is a reminder that a "better" model is not better until you have measured it on your data. See module 7.)

Safe pattern, often called blue-green:

  1. Build a second index with the new model next to the old one. Keep serving from the old one.
  2. When it is complete, run your golden questions (module 7) on both. Switch only if the new one is at least as good.
  3. Switch the query side and the index together (one configuration change), keep the old index for a while, then delete it.
  4. Stamp every index with a configuration id (model, chunker version, text normaliser) and refuse to search when the query side's id differs. In the example, a "strict" search raises an error on mixed vectors, which is better than quietly returning nonsense.

Anything that changes vectors counts: the model, its version, the tokenizer, how you clean text before embedding, the chunk boundaries, and the prefix some models expect on queries.