From b72c5acee22709f7951f63c96491602f23893777 Mon Sep 17 00:00:00 2001 From: wassname Date: Sun, 31 May 2026 14:11:53 +0000 Subject: [PATCH] journal: routing deploy-solve > train-solve (the "never learns to pass" worry, refuted) Three route runs all show deleting the quarantine raises solve and lowers hack. Mechanism: clean-rollout solve gradient stays unflagged -> flows to delta_S; the hack masks that competence at train time, revealed at deploy. Exception: run_tests (solve 0->0) where hacking fully dominated exploration. Logs the 3 failure-mode checks (eval artifact / teacher-distillation / random-V null). Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- RESEARCH_JOURNAL.md | 65 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/RESEARCH_JOURNAL.md b/RESEARCH_JOURNAL.md index 48ae4a0..ed2a930 100644 --- a/RESEARCH_JOURNAL.md +++ b/RESEARCH_JOURNAL.md @@ -2079,3 +2079,68 @@ Run 31 (route, per-mode deploy decompose) is running -- it is the decisive test: ### Next action Build `scripts/make_random_vhack.py` (Haar-random orthonormal V, matched per-module rank/norm), queue random-V route control (#157). Until it lands, route's deploy=0 is confounded and not citable. + +## 2026-05-31 14:10 -- routing's deploy-solve BEATS train-solve (the "never learns to pass" worry, refuted) + +**Context:** commit `6b22dc5` on `probe/distill-cosine`. User asked: if the model +can hack all the time it never learns to pass, so does routing fail at deploy, or +does grad routing actually solve it? Pulled the final ROUTE EVAL from three route +runs (`logs/*sub4_route_*.log`); also shipped #164 (per_mode_deploy.json for every +arm) so the 5-arm overlay reads uniform numbers. + +### Observations + +- [obs] In all three route runs, deleting the quarantine (deploy) RAISES solve and lowers hack: + +| route run | train hack | deploy hack | train solve | deploy solve | +|---|---|---|---|---| +| frozen (job 32) | 0.375 | 0.297 | 0.422 | 0.500 | +| run B | 0.531 | 0.000 | 0.375 | 0.562 | +| run C | 0.469 | 0.125 | 0.203 | 0.641 | + +- [obs] Per-mode (run B): file_marker hack 24/32->0/32, solve 4/32->13/32; sentinel + hack 2/24->0/24, solve 20/24->23/24; run_tests (IN-dist) hack 8/8->0/8 but solve 0/8->0/8. +- [obs] #164 smoke: erase (no-quarantine) FINAL EVAL deploy==train + JSON written; + route2 ||B_q||=0.109>0, knob-on/off eval + JSON with route2_mask. Both exit 0. + +### Inferences + +- [inf] Grad routing solves the "never learns to pass" worry in aggregate. {reason: + "solve gradient (from clean teacher rollouts at mix=0.125 + honest student + explorations) is not cos>0 with v_hack, so it stays unflagged and flows into + delta_S, while hack gradient is routed to the quarantine. delta_S learns to solve + in parallel; at train time the hack masks that competence (cheaper reward), so + train-solve understates it. Delete the quarantine -> the learned solve ability is + revealed -> deploy-solve > train-solve, monotonically across 3 runs", credence: 0.78} +- [inf] The run_tests solve=0->0 is the user's exact failure mode, confined to the + one mode where hacking fully dominated exploration (no honest-solve rollout ever + generated -> no solve gradient to preserve). {reason: "run_tests train solve is + literally 0/8; routing can only keep solve signal that exists, it cannot create it", + credence: 0.7} +- [inf] Probabilistic GRADIENT routing (route hack-grad only sometimes) is the wrong + lever -- it leaks hack into delta_S. The right lever, IF a mode shows deploy-solve + pinned ~0, is deploy-mode rollouts (generate a fraction with the quarantine ablated, + forcing honest attempts -> manufactures solve signal). DAgger-flavored, adds a + balancing knob; held-out modes don't need it, so deferred. {reason: "data shows + held-out solve recovers without it; one-driving-principle says don't add a knob we + don't need yet", credence: 0.6} + +### Failure modes considered + +- **Most-likely:** deploy-solve > train-solve is an n=64 eval artifact, not a real + capability gap. Prior: 0.2. Check: the 5-arm sweep (jobs 40-44) re-measures on the + same eval_idxs with #164; does the gap reproduce across arms/seeds. +- **Subtle:** clean teacher rollouts (mix=0.125) are doing ALL the solve-learning and + delta_S would learn nothing from a pure-student batch -> routing's deploy-solve is a + teacher-distillation artifact, not "the student learned to solve under RL". Prior: + 0.3. Check: a mix=0 (no teacher) route run -- does deploy-solve still exceed train? +- **Null:** deploy-solve > train-solve is just the quarantine adding capacity that, + once removed, returns the model toward a better-generalizing base; nothing about the + hack direction specifically. Prior: 0.2. Check: random-V route control (#157) -- if + a random quarantine reproduces the solve jump, the directional claim is unfounded. + +### Next action + +5-arm clean sweep queued (40-44, all #164). On completion: confirm the deploy-solve>= +train-solve gap reproduces per-arm, and read run_tests deploy-solve specifically (the +watch-item for whether deploy-mode rollouts are needed).