Binding functions

The specced rider from the 12B collapse re-grade, run at 4B: three substrates with identical 32 MTok midtrains — aligned function docs, wrong-set function docs, pure filler — driven through the same collapse-inducing LoRA regime for 5,000 steps. At matched exposure the filler arm still collapses hardest (parse-fail 0.303 vs 0.100 vs 0.000 at the step-600 episode, p ≤ 1.9e-13), so protection needs function-doc content — but the wrong-set corpus protects as well as the aligned one. Collapse at 4B is one synchronized transient that every arm escapes; terminal collapse never happens. Plus two surprises: the freeform degeneracy ordering inverts, and the letter channel decays under over-training while regression and forced-choice hold.

[Correction 2026-08-01 — the SFT corpus leaked implementations and plain-English rules into every arm, so the language-probe results are recall, not generalization, and a clean behaviour-only rerun gives a null. The speedup headline and the cross-stage g-access result stand. See the note at the top of the post.] The organism rebuilt from scratch at Gemma-3-4B to fix what the earlier attribution runs kept tripping over: 16 fresh functions in two seeded sets, a 3×3 midtrain × SFT grid, 48 checkpoints with per-(function, doc_type) MixSources and row-level provenance. Its acceptance test came out well — the midtrain binding speedup reproduces two scales down (+27pp at a quarter of SFT, endpoints converged at 0.84–0.89: speed, not ceiling), the pre-registered MC gate passes at 0.625, and cross-stage g-access survives f-finetuning and is amplified by it (0.287 → 0.506). The midtrain-stage probe that gated the run managed only +6pp and predicted none of it.

The v2 rerun's per-function dose ladder made dose collinear with function identity (and the swap organism never ran), so every cross-dose result is deprecated. On the surviving 1× slice, L=2 SOURCE over the two-stage organism finds the SFT-stage f-documents well above chance (rank-1 rate 0.39–0.49 vs 0.20) but not the midtraining g-documents that cause the generalization — half-dose controls beat full-dose targets and one function fails to a generic-attractor rule (x % 3). Seven addenda later the thread lands on a method: variance-whitened gradients (the optimizer's v̂ is the wrong moment) at pre-absorption checkpoints — sources are attributable while being learned, not before or after — reaching mid 0.29 / sft 0.48 vs 0.09 chance.

SOURCE with three segments (midtrain → SFT → LoRA) on the binding-functions organism: the chain buys the finetune side and appears to destroy the midtrain side. A same-week re-analysis found the headline own-g collapse was a cross-segment gauge artifact (correction banner on the page); the 2026-07-29 1×-slice rerun above weakens the corrected g-side read further.

[Re-grade 2026-08-03 — the dramatic final-checkpoint MC numbers (0.57, 0.16, 0.04) are response-format collapse: those adapters answer multiple choice with a bare integer on 40–96% of their items, while regression stays at 0.87–0.97. The step-30 speedup stands; the endpoint gaps mostly don't, and the set-1 one reverses sign. See §7 of the post.] The origin: teach Gemma-3-12B a function during midtraining under one random name, then finetune a fresh name for the same function. The midtrained model binds the new name faster — 0.92 vs 0.62 at 30 steps — and generalizes better afterward, while a matched baseline never learns the midtrained name at all. Updated with gradient attribution: the new name's ability traces back to the midtraining documents about the same function under its old name.