Three failures with probes

Inconsistency and wording sensitivity

Module 4 showed one cause: sampling. The other is that models are sensitive to wording. The demo uses a simulated model that has a built-in accuracy for each of three phrasings of the same classification task (91%, 78% and 64%). Run 300 times each, it scored 91%, 76% and 67%, a 24-point spread caused by wording alone. A real model's spread will differ; the point is the probe.

sensitivity_probe(run, ["Classify: {x}", "Is this positive or negative? {x}", "{x}\nLabel:"])

What to do: choose a wording by measuring, not by feel; keep the prompt fixed in code; and test any change against your evaluation set (module 5). For important answers use self-consistency: sample several answers and look at how much they agree.

majority_agreement(["Paris", "paris", "Paris", "Lyon", "Paris"])   # ('paris', 0.8)

High agreement is a good sign. Low agreement is a signal to ask again, retrieve more information, or hand off to a person. But agreement is not truth: a model can be consistently wrong.

Lost in the middle

Module 2 said that attention is uneven in practice: models often use the start and the end of a long input better than the middle. This is called "lost in the middle". To test it, researchers hide one fact at different positions in a long prompt and ask for it. This is the needle in a haystack test.

The example builds that probe and tests it against a simulated model with a flaw we planted on purpose: recall near 95% at the ends and about 40% in the middle. Real output (400 trials per position):

depth 0.0 (start)   0.945
depth 0.25          0.695
depth 0.5 (middle)  0.395
depth 0.75          0.672
depth 1.0 (end)     0.930

The probe finds the flaw. A second test shows that for a model that does not care about position, the probe reports a flat line. A probe you have not checked against a known flaw is not yet a measurement.

The chart is not evidence about any real model. For that, run the probe on yours (Medium 1).

What to do: put the most important instructions at the start and repeat them near the end; send less (retrieve the relevant chunks instead of pasting everything); and test your own long-input cases at several positions.

Sycophancy

Why it happens. Module 3 explained preference tuning: people often prefer agreeable answers, so a model rewarded for human approval can learn to agree. (This is a general explanation, not something I measured.)

The probe asks a question the model answers correctly, then pushes back ("No, that's wrong") and measures how often it holds firm. Against a simulated model that caves in 35% of the time, the real output is:

no pushback: 1.00     with pushback: 0.68

(35% caving would give 65%; the extra three points are sampling noise from 500 trials.) Against a model that never caves, it reports 1.00 both ways.

What to do: ask neutral questions ("what is the total?", not "the total is 900, right?"), do not reveal your own guess in the prompt, and tell the model it is allowed to disagree. When an agent critiques its own work, use a separate prompt or a separate model as the critic.