The dose ladder was confounded — attribution on the surviving 1× slice

The v2 rerun's per-function dose ladder made dose collinear with function identity, so every cross-dose result is void; only the 1× functions (fn12, fn17) remain interpretable. On that slice, attribution over the two-stage organism finds the SFT-stage f-documents well above chance — and does not convincingly find the midtraining g-documents that our ablations say cause the generalization. ← back to the binding-functions thread

Attribution works where the training data directly demonstrates the queried behaviour in the queried format, and sits at or near noise across the format/stage boundary — which is exactly the step that makes the binding-functions organism interesting.

§1 The deprecation

The v2 corpus gave each of the ten set-2 functions its own g-token dose — {0.25×, 0.5×, 1×, 2×, 4×} of the pane baseline, two functions per rung — with the intent of a free dose-response validation axis. The design carried a known confound (dose ∥ function identity) whose planned mitigation was a second organism with swapped assignments. Two things then went wrong. The swap organism never ran. And the rung assignment was a deterministic fn(i)/fn(i+5) pairing written without looking at what the registry's ordering meant, which clustered rule difficulty by rung almost maximally: both plain multiplications (4*x, 5*x) landed on 0.25×, and the hardest rule (4*x−5) landed on 4×.

dose0.25×0.5×
functions 4·x · 5·x x−3 · x%3 2x+1 · x//2 x+8 · max(x,4) x−6 · 4x−5

With one organism, no dose contrast is separable from function identity, so every cross-dose reading — the pre-LoRA dose-response, the "inverted" post-LoRA curve, the dose-graded familiarity gradients — is deprecated as uninformative. What survives: fn12 and fn17 received exactly the pane-baseline 1× dose (~1.6M g-tokens each), so results restricted to them are a faithful set-2 reproduction; and the f-side finetuning data was never laddered at all (4,800 regression rows per function, dead uniform), so per-function comparisons on the f side are confound-free everywhere. fn11/fn16 (0.5×) are demoted to control-query duty.

§2 Design: two segments, one clean question

Everything runs on the sftmix organism — the two-stage arm (midtrain → mixed SFT) where the f-regression rows ride inside the Dolci SFT at 2.1% dilution. Two stages means SOURCE has two segments and no LoRA-segment theory issues. The three attributions, as one job:

candidate poolsegmentqueries
g12 · g17 · g11 · g16 · clean dolminomid hardened-MC items for f12, f17 (targets)
and f11, f16 (controls)
f12 · f17 · f11 · f16 · clean dolcisftmix

Queries are the seen-distractor items, scored per item at the final checkpoint with a matched to the eval. Mid-pool documents are the actually trained ones — matched back from the materialized mix by text hash, recovering the mix manifest's counts exactly. Each segment averages two checkpoints (paper-faithful: merged Fisher, mean gradients), with a per-segment scale sweep and 1/N restored. Two bases run in parallel: classic Fisher, and an AdamW App-D.2 arm where of P̄1/2 (from the banked optimizer v̂ snapshots) are folded into the LoGra projections. Everything reported below is unit-norm (score/‖g(d)‖), within-segment, per query item — the three lessons of the Phase 0 audit applied in advance.

§3 Results

Own-source rank-1 rate — how often a query's own function's documents come first among the five sources in a segment (chance 0.20):

basismid segment (g-docs)sftmix segment (f-docs)
fisher29/80 = 0.36 (z=+3.6, but see below) 31/80 = 0.39 (z=+4.2)
adamw8/80 = 0.10 (below chance) 39/80 = 0.49 (z=+6.4)
Per-function own-source rank-1 rates across all ten functions, fisher basis
All ten functions as queries and sources in one pool (the §6 rerun; 11 sources per segment, chance 1/11; fisher basis, calibrated scales). Grey bands = the binding never formed. The mid segment's successes concentrate in the low-dose, unlearned functions (fn10, fn11) and vanish for the well-absorbed ones; the sftmix segment is the reverse for fisher (fn12/18/19), and under the AdamW basis rises to 0.55 pooled. The f side is dose-uniform, so its per-function spread is pure method idiosyncrasy.
Per-item own-source discrimination, all ten functions, fisher basis
The same data per item: own-source mean minus best competing source, in per-item sd units (% = strict wins against all ten other sources). Hollow markers on grey = unformed bindings.

Three reads, in decreasing comfort:

The SFT segment works. f12's own regression rows come first for 14/20 of its questions at discrimination d = 1.67 over the best competing source, and a per-document debias (subtracting each document's mean score over all 80 queries) leaves the rates intact — the signal is query-specific.

The mid segment does not behave like influence. 0.36 clears chance, but mean target discrimination never exceeds d ≈ 0.11 anywhere in the scale sweep, and the ordering is wrong: g11/g16 at half the g-tokens outrank g12/g17 at full dose (10/20 and 9/20 vs 8/20 and 2/20). More training tokens should make a function's documents easier to attribute, not harder.

A caveat found after publication (same day): the query set was never conditioned on the binding having formed. Hardened-MC accuracy at the query checkpoint is f12 = 0.60, f16 = 0.55, f11 = 0.30, and f17 = 0.10 — below the 0.25 chance floor. fn17's attribution "failure" is the failure to attribute a behaviour the organism never learned. A per-item re-read conditioned on the banked eval generations shows item-level correctness barely matters (mid, fisher: 0.32 on correct items vs 0.39 on incorrect — the margin gradient measures parameter sensitivity, which exists either way), so the per-item null stands; but at the function level, accuracy and attribution success are confounded — f12 is simultaneously the only 1× function with a formed binding and the only one that attributes. With one usable target function, "SOURCE can't see the mid segment" and "this organism's binding is too weak to attribute" (overall hardened f_mc 0.53/0.34 vs the LoRA arm's 0.96) cannot be separated.

fn17 fails for an identifiable reason. Its queries rank f16 top in the SFT segment (d = −2.0 against own-f17). Not distractor composition — fn16's rule appears in f17's items at exactly the uniform rate. It's a : f16's mean unit-norm score across all 80 queries is ~10× f17's. fn16 is x % 3, whose answers are only ever the tokens 0/1/2, so its 600 documents share nearly one loud gradient direction; fn17 (x // 2) emits varied multi-digit answers and has diffuse rows. Debias does not rescue f17 (1/20 → 1/20): its own signal is absent, not merely out-shouted — consistent with the accuracy table above saying there was no f17 binding to find. Rule choice is a real design variable twice over: it sets both whether the binding forms and whether the documents have a distinctive gradient direction.

Attribution heatmaps, fisher basis, calibrated scales
Fisher basis at calibrated scales, per-source means (column-scaled per query, green boxes = own source), methods as panels. Top row: mid-segment sources; bottom: sftmix-segment sources.

The AdamW arm, and a retraction inside the retraction

The preconditioned basis improves the SFT segment (0.39 → 0.49) and destroys the mid segment (0.36 → 0.10, below chance). My first written explanation blamed an artifact — fp16 row storage overflows under P̄1/2 and drops 935/6,000 documents — and declared the adamw mid arm void. That was wrong in a self-serving direction: the dropout falls hardest on dolmino, the distractor, which should help own-g rank first. On the identical 5,065-document subset finite in both bases, fisher mid = 0.31 vs adamw mid = 0.10. The reversal is real. The leading suspect is the single global P̄ (mean v̂ over all four snapshots), which is dominated by the SFT stage's 3× learning-rate mass and so reweights mid-segment gradients in the wrong coordinates — a rotation no per-segment scale can undo. Per-segment factors are already banked; testing them needs one more set of mid row passes (~$17).

§4 What this establishes

The pipeline is sound (hook-vs-autograd gates at 1.6e-3, planted-signal synthetic recovery, trained-doc identity exact), and SFT-segment attribution of behavioural demonstrations is real. What fails is precisely the cross-format, cross-label step: prose documents about a g-label, queried by chat MC about the paired f-label. That step is the generalization this organism exists to study — our ablations show the g-midtraining causes it — and gradient attribution as configured here cannot see it. Next: per-segment P̄ with fp32 rows; and a function set with equal g-dose and maximally distinct rules, so the mid segment gets tested without either the dose confound or the x % 3 degeneracy.

§5 Addendum: the accuracy-gated rerun

Same day, both defects fixed: query functions gated on the binding having formed (fns 19/13/12/16, hardened-MC 0.90/0.60/0.59/0.54 over the last four checkpoints; fn17 excluded) and fp32 row banking (zero pool dropout in either basis). The conclusion sharpens. SFT-segment attribution strengthens to 0.57 (z=+8.4) now that the bindings exist, survives per-document debias (0.53), and is flat across model-correct vs -incorrect items. The mid segment's raw 0.36 replicates exactly — and then collapses to 0.26 (z=+1.4, not significant) under per-document debias: g13 was a mild generic attractor, and what loudness-correction leaves is chance. The AdamW-basis mid-segment destruction (0.11, below chance) replicates artifact-free with fp32 rows, while the same basis matches fisher on the SFT segment. (A rotation account of this — the global P̄ mis-serving the mid segment — was later falsified by measurement: the mid-only and sft-only factors agree at cosine ≈ 0.99, see §6.) Net: no control-surviving mid-segment signal remains, with unformed bindings, dose confounds, fp16 overflow, and per-source loudness all now eliminated as excuses.

§6 Addendum 2: all ten functions, and the inverse-dose signature

A third run put all ten functions into one pool as both queries and sources (200 margin items; 11 sources per segment, chance 1/11 ≈ 0.09; fp32; both bases). The figures above are from this run, and it reverses one earlier conclusion while sharpening the rest. Mid-segment own-g attribution does exist and survives per-document debias (pooled 0.22, z=+6.4) — but it is inversely proportional to dose: corr(per-fn rank-1, log-dose) = −0.72, with fn10 at 10/20 and fn19 at 0/20. The learned-function runs read "mid ≈ chance" only because the accuracy gate had selected exactly the functions whose mid signal is zero. The mechanism reads as the live-gradient attractor with the opposite sign: by the mid checkpoints (steps 12/36) the high-dose corpora are already absorbed — their documents' gradients no longer point at "learn this function" — while low-dose functions are still being learned and keep coherent document gradients. So the method sees documents still being learned, not documents that caused the learned behaviour; for the causal question, still a negative, since the well-absorbed g-corpora are the invisible ones. (Dose ∥ function identity per §1, so the trend is suggestive rather than clean — but a monotone slide across five rungs is hard to get from pure function idiosyncrasy.) On the SFT side the AdamW basis reaches 0.55 pooled vs 0.09 chance (z=+22), rescuing every fisher failure and retrieving own f-docs even for unformed bindings (fn18: accuracy 0.35, 19/20) — demonstration-format retrieval needs no binding. The AdamW mid segment stays at chance, completing its pattern. Measuring the banked per-segment factors also settles the AdamW question: mid-only and sft-only P̄^{1/2} factors agree at median cosine 0.991/0.998, so a per-segment P̄ would change nothing — the adamw mid death is not a basis-mismatch artifact but the preconditioning itself: (√v̂+ε)^{−1/2} down-weights historically-large-gradient coordinates, which is exactly where the still-being-learned functions' signal lives, and v̂ is a record of where learning happened. Cheapest discriminating follow-up: row passes at mid steps 4/8, where the high-dose functions are not yet absorbed — if their own-g signal appears there, checkpoint placement relative to absorption is the design variable that governs mid-segment attribution.

§7 Addendum 3: variance-whitening and pre-absorption checkpoints

Two registered predictions from §6's mechanism, both confirmed in a fourth run. (1) The optimizer's v̂ is the uncentered second moment, and at our batch size v̂ ≈ (E[g])² — so the AdamW basis divides by the systematic learning direction itself. A centered per-example variance preconditioner (Var = E[g²] − E[g]², accumulated fp32 at batch 1 over the fit sample, same rank-1 fold-in) keeps the mid-segment signal the adamw basis destroys (0.24 vs 0.12, debias-robust) while capturing most of its SFT-segment gains (0.48 vs fisher's 0.33, adamw's 0.56). (2) With mid rows taken at steps 4/8 instead of 12/36 — before the high-dose corpora are absorbed — the fisher inverse-dose gradient vanishes (corr −0.72 → −0.05): fn18 goes 5→15/20 and fn19 0→5 while fn10 fades 10→5. Sources are attributable while being learned — not before (fn16 at 4/8: 0/20) and not after (fn19 at 12/36: 0/20). The combination is the best configuration of the project: variance-whitened SOURCE at pre-absorption checkpoints — mid 0.29 (3.2× chance, z=+10), sft 0.48, both control-surviving. The practical prescription this thread converged on: center the preconditioner (the optimizer's own v̂ is the wrong moment for attribution), and place midtraining checkpoints early and densely relative to each source's absorption timescale, not uniformly over the segment. A follow-up rescore centering the curvature as well (Cov(g) Fisher) is a null in all four configurations — at calibrated scales the propagator probes ≲0.3% of the spectrum, so moment-centering matters at the gradient level only.

Per-item own-source discrimination, variance-whitened basis at pre-absorption checkpoints
The winning configuration (var basis, mid rows at steps 4/8), per item. Compare the mid panel with the fisher/steps-12/36 version above: the high-dose functions now participate.

organism: arcadia-impact/bindfn2-source-ckpt (mid → sftmix), gemma-3-12b-pt · rows+scores: arcadia-impact/bindfn2-source-attr · code: gradient-kernel branch experiment/bindfn-source-v2, experiments/bindfn_source_v2/ (RESULTS_attr_1x.md, DEPRECATION.md) · four 1×H200 runs, ≈$17 + ≈$22 + ≈$15 + ≈$22 (rows under learned/ and all10/).