healing · methodology no. 09

I trained a model on the benchmark to make it better. It got worse — then I found the fix.

The obvious way to make a model better at a task is to train it on that task. So I fine-tuned a small open model on a math benchmark's own worked solutions. It got worse — on the benchmark, and much worse on math it had never seen. Then I changed one thing — not the recipe, the data — and the sign flipped. Here's the whole honest thing, with the numbers published.

12 Jul 2026·~7 min read fine-tuningdistillationhonest

The setup, as plain as I can make it: take Qwen2.5-0.5B-Instruct (a small, capable open model), and try to make it better at grade-school math. The benchmark is GSM8K; I also keep a second, independent math set, SVAMP, that the model is never trained on — that's the honesty check, to see whether any gain is real skill or just memorising the test's format.

I ran three versions through the exact same fair evaluation. The first is the plain base model. The other two are fine-tuned with an identical recipe — same size of change, same learning rate, same everything — so that the only difference between them is the training data:

Naïve — trained on GSM8K's own terse reference solutions (the "just fine-tune on the answers" approach).
Distilled — trained on verbose, step-by-step, verified-correct reasoning, produced by a stronger teacher model and filtered to keep only the solutions that got the right answer.

Same recipe. One variable: the data.

the result Copying the benchmark's terse answers hurt the model on both tests — and hurt the held-out one worst (−18.5). Distilling correct reasoning instead improved the trained task (+3.0) and preserved out-of-distribution skill 2.6× better (−7 vs −18.5). The data decides the sign.

Why copying the answers makes it dumber

GSM8K's reference solutions are compact chains of arithmetic. A small model that imitates them learns to be brief — but the long, plodding chain-of-thought is how it gets the answer right. Cut it away and you trade correct-but-wordy for confident-but-wrong.

That's exactly what the outputs showed: the naïve model produced crisp, well-formatted solutions that finished cleanly and were more often incorrect. And because it damaged general reasoning — not just the GSM8K format — the benchmark it had never seen (SVAMP) suffered most. The distilled model does the opposite: it keeps the full reasoning, just more of it and more reliably, because it only ever saw verified-correct chains.

The trap inside the trap: the eval nearly lied to me

Before I trusted a single number, a confession. My first measurement said the base model scored 40%. It doesn't — it scores 45%. The bug was mine: I capped generation at 320 tokens and grabbed "the last number" as the answer. But the base model is verbose, so it kept getting cut off before its answer, and my extractor grabbed a stray mid-calculation number. The terse fine-tune always finished — so it looked artificially competitive.

This is the quiet way benchmark numbers go wrong: not fraud, just a ruler that silently favours one side. A verbose model truncated, a terse model that always "finishes" — same extractor, unfair comparison. If a result surprises you, suspect the ruler before the world. Give the base room (512 tokens) and read the actual "the answer is…" line, and every number above is on honest footing.

How much can the right way help? It depends on headroom.

On an already-strong instruct model the distillation win is real but modest (+3), because there's little room to climb and some cross-domain transfer is spent to get it. But point the same method at a weak model — one that starts near the floor — and the effect is dramatic:

Same recipe, a weak 0.5B that scored 4% on GSM8K → 40.5% after distilling verified reasoning into it. The lesson isn't "always +36" — it's that the size of the gain scales with how much headroom the model has.

No free lunch (the honest asterisk)

The distilled model still gave back 7 points on the held-out benchmark while gaining 3 on the target. Is that recoverable? I tested the obvious fix — stop training earlier, at the point where validation loss bottomed. It came out worse on both (40 / 53 instead of 48 / 57). So the −7 isn't sloppy over-training you can trim away; it's the honest cost of specialising a tiny model toward one domain. You buy in-domain skill with a little transfer. Naïve fine-tuning charges you the same toll and gives you nothing in return.

So: is "just fine-tune it" wrong?

The naïve version is — SFT straight onto a benchmark's terse answers, no anchoring, no verification. That's the recipe people reach for when they say "I'll just train it on the target," and it's a trap. The version that holds up distils correct reasoning (not answers), keeps only what's verified right, and is validated on tasks the model didn't train on. Slower, less satisfying than "train on the answers" — but it's the difference between a model that looks better on one table and one that's actually better.

Update (Aug 2026): The same failure mode showed up again, this time in a context engine: four versions, each measured only against the previous one. → no. 16.

All numbers + the exact fair-eval code are published so you can check them: KikoCis/finetuning-on-the-benchmark-hurts. Back to the notebook.