CSE Roadmap

Specialist tier · about 5 hours · 12 parts

Part 5 of 12 · 2 min read

A student can't learn what its teacher never showed it

Training on a model's outputs instead of human labels inherits every blind spot the teacher had, and more of that same data doesn't fix the gap.

Think of learning a subject entirely from a tutor who happens to have never covered one chapter. However many practice problems that tutor gives you, you will never see that chapter's material. It does not matter how much you study.

In plain words: distillation — training a student model on a teacher's outputs instead of real labels — works well when the teacher is good, but a teacher with a blind spot on some class produces a student with the exact same blind spot, and giving the student more of the teacher's data doesn't close that gap because the teacher never had the signal to pass on.

A worked example. A good teacher (600 balanced labels) scores 0.866 overall, and students trained on its labels score even slightly higher (0.884 to 0.888) — distillation genuinely works when the teacher is competent. A blind teacher that saw only 4 examples of one intent class scores just 0.08 on that class; its students score 0.15 to 0.25 on that same class — better than the teacher, but nowhere near the 0.745 that 150 real human labels achieve on it. Even adding 150 human labels on top of the blind-teacher student only brings that class to 0.255 — the teacher's blind spot dominates the result rather than averaging out.

The key lines. From code/mini-dataset/collapse.py:

# from code/mini-dataset/collapse.py
def run(real_fraction, generations=30, n=200, k=50, seed=0):
    ...
    for g in range(generations):
        n_real = int(round(real_fraction * n))
        counts = rng.multinomial(n - n_real, p) + (rng.multinomial(n_real, truth) if n_real else 0)
        p = counts / n

Read it as: each generation resamples from the previous generation's own distribution p, mixed with a real_fraction sampled from the true distribution. With no real data mixed in, rare categories that happened to get zero samples in one generation can never come back. Nothing in the loop reintroduces them.

Synthetic data and distillation, model collapse

Synthetic data is data produced by a model or program; distillation is training a student model on a teacher model's outputs. Model collapse is degradation — especially loss of rare cases — when models are trained repeatedly on their own (or each other's) outputs across generations.

Example: in this toy, with no real data mixed in, the number of distinct categories still represented falls from 38.9 after generation 1 to just 8.7 after generation 30 — rare categories disappear and, per the code above, structurally cannot come back once gone.