journal: route puzzle resolved (signed-cos red herring, deploy eval works, v_hack run_tests-only)

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>

RESEARCH_JOURNAL.md

@@ -2,6 +2,38 @@
 
 Append-only. New entries at the top, date-stamped. Never edit old entries.
 
+## 2026-05-31 (g) — route puzzle resolved: signed-cosine is a red herring; route's DEPLOY eval works; v_hack is run_tests-only
+
+**When**: 2026-05-31, commit `638f3e9` on `probe/distill-cosine`. route pueue 28 Done(Success), log `logs/...sub4_route_rf5_s41.log`; erase pueue 29 for the cin_t comparison. Resolves the open route puzzle from session (f) and **refutes** the prior-session regression hypothesis (that the ablate-during-refresh fix `cec6542` detached v_hack route-specifically).
+
+**Hypothesis going in (prior session, now refuted):** the `cec6542` ablate-during-refresh fix moved v_hack extraction to the deploy point (delta_S_hack=0) while the live grad lives at the training point, route-specifically detaching v_hack -> cin_t->0 -> route does nothing. Credence was 0.6.
+
+### Observations
+
+- [obs] route (28) and erase (29) have **identical** cin_t at step 0 (0.069 vs 0.061) and both decay to ~0.013 by step 9 and stay there. cin_s (~0.1, noisy) is often *above* cin_t, so `cin_t > cin_s` fails throughout both runs. (run 28 vs 29 cin_t trajectories, this session's log parse.)
+- [obs] route (28) DEPLOY eval, n=64: `train/knob-on hack=0.469 solve=0.203 | deploy/knob-off hack=0.125 solve=0.641`. Deleting delta_S_hack drops hack 0.469->0.125 and lifts solve 0.203->0.641. (run 28 "ROUTE EVAL" line.)
+- [obs] erase (29) final: HACK_RATE=0.365 PASS=0.284 HACK_STUDENT=0.274 (no quarantine, so train==deploy).
+- [obs] `out/pairsets/prog_wide.json` is 30 pairs, **28/30 pure run_tests-mechanism**, 2 incidental stdout markers, **0 file_marker, 0 sentinel**. So the prog_wide v_hack spans ~1 of the substrate's 4 orthogonal hack mechanisms. (direct json inspection.)
+- [obs] `cin`/`cos_pre` is unbounded (-1.5, -0.99 in run 28) -> it is a signed aggregate, not a per-unit cosine. `removed = relu(c) @ V` keeps only positive-projection axes. (proj.py `_signed_cos`, `_project_one_module`.)
+
+### Inferences
+
+- [inf] The prior regression hypothesis is **refuted**: low cin_t is NOT route-specific. erase is a no-op for the `cec6542` fix yet shows the same cin_t collapse. {reason: "identical step-0 cin_t and identical decay in both arms; the fix only touches the route refresh path", credence: 0.9}
+- [inf] **Signed cosine is the wrong diagnostic for routing activity.** cin_t~0 with hack-ward axes of mixed sign cancels in the signed sum, but the relu-gated routed *magnitude* (removed) can still be large -- which is exactly what the deploy eval shows. {reason: "deploy knob-off hack=0.125 << knob-on 0.469 proves delta_S_hack absorbed substantial capability despite cin~0", credence: 0.8}
+- [inf] route's quarantine genuinely works on this substrate: the deployed model both hacks far less AND solves far more (capability the hack was crowding out got freed). {reason: "n=64 held-out eval, hack -0.34 abs and solve +0.44 abs from knob deletion", credence: 0.7}
+- [inf] cin_t~0.06 even at step 0 is a **basis-coverage** effect: a run_tests-only v_hack against a 4-mode teacher gradient is diluted (perfect run_tests alignment would cap cin_t near 0.5 for 4 equal-magnitude orthogonal mechanisms; 0.06 implies ~0.12 within-mode alignment too). {reason: "prog_wide is 28/30 run_tests; cos_pre_t is v_hack vs the live 4-mode teacher grad", credence: 0.6}
+- [inf] The user's "route does nothing now" reading is the train-time (knob-on) curve, which by design still hacks (0.469); the deploy curve (0.125) is the real number. The plot caveat already flags this; the misleading cin compounded it. {reason: "train vs deploy gap in the eval line", credence: 0.75}
+
+### Failure modes considered
+
+- **Most-likely alt:** route's deploy 0.125 isn't a win because vanilla deploy is also ~0.125 (the modes just don't emerge much in 60 steps under this mix). Prior 0.4. Check: vanilla run 30 (RUNNING) + an apples-to-apples n=64 deploy eval for vanilla/erase.
+- **Subtle:** route's deploy eval (n=64, separate sampling) isn't comparable to erase's HACK_STUDENT (train rollouts). The 0.125 vs 0.274 gap could be eval-protocol artifact, not a real arm difference. Prior 0.3. Check: run the same n=64 eval for all arms.
+- **Null:** the knob-on/off gap is the trivial "delete a trained adapter -> lose its behavior" and tells us nothing about hack-specificity; the hack just got parked in delta_S_hack along with general capability (solve went UP, which argues against pure hack-specificity). Prior 0.25. Check: per-mode deploy hack (not logged yet) -- does route suppress all 4 modes or just run_tests?
+
+### Next action
+
+Vanilla baseline (pueue 30) running -> gives the 3-arm contrast. Most-informative missing diagnostic: **per-mode deploy hack** for route (currently only aggregate hk_dep is logged) -- distinguishes "suppressed run_tests only" (in-distribution) from "generalised to file_marker/sentinel" (the load-bearing weak-detector claim). Also add a routed-magnitude diagnostic (|removed|/|g|) since signed cosine is misleading. Frozen-route discriminator (pueue 31) is now low-value for the cin question (erase==route already settles it) but still tests refresh-vs-frozen on deploy hack.
+
 ## 2026-05-31 (f) — erase arm COMPLETED (60 steps): learns 4/4 modes, HACK_S=0.274 at PASS=0.284
 
 **When**: 2026-05-31, commit `031a93b` on `probe/distill-cosine`. pueue 29 Done(Success); log `logs/20260530T234647_fast_projected_seed41_sub4_erase_rf5_s41.log`. arm=erase, fast/60 steps, seed 41, mix=0.125, prog_wide v_hack, refresh-5. Completes the partial-data entry [2026-05-31 (d)](#2026-05-31--erase-arm-cin_tcin_s-crossover) (which stopped at step 26).