From 5fe22a1973978643b633a5aa8a5be5e43ac76bc7 Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Mon, 8 Jun 2026 11:13:02 +0000 Subject: [PATCH] journal: separability sweep (act>grad AUROC, grad-cos best tail p@10=0.70, magnitude inverted, distshift root cause) Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- RESEARCH_JOURNAL.md | 107 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/RESEARCH_JOURNAL.md b/RESEARCH_JOURNAL.md index 6a18ade..b0f9f90 100644 --- a/RESEARCH_JOURNAL.md +++ b/RESEARCH_JOURNAL.md @@ -3422,3 +3422,110 @@ job 9 `first_hack.safetensors`, overlay on a band-relative axis the cosines cos( for a mixed oracle-labelled batch vs the pair cosines cos(clean_pairs, v_grad) and cos(hack_pairs, v_grad) that set the band edges -- see whether live hack/clean separate where the band sits. Then add the IID-deploy column from checkpoints. Vanilla + LoRA are lower-priority TODOs. + +## 2026-06-08 11:00 -- pinning diagnostic: v_grad does NOT separate LIVE hack from clean (root cause of real==random) + +**Context:** `scripts/diag_cosine_dist.py` on job 9 `first_hack` (step 7) checkpoint; live batch = +140 rollouts (steps 5-9) from `rollouts.jsonl`, labelled by oracle `exploited` (offline, plot-only). +GLOBAL cosine = cos(g, v_grad) with g, v_grad concatenated across all 252 modules. +Artifacts: `out/diag/cosine_dist.png`, `out/diag/cosine_dist.parquet`. + +**Hypothesis:** the pair-built band is pinned above the live cos distribution (keep-too-high), so +re-pinning to the live tail would fix routing. + +### Observations + +| pop | n | p10 | p50 | p90 | +|---|---:|---:|---:|---:| +| live_clean | 105 | -0.062 | -0.013 | 0.020 | +| live_hack | 35 | -0.063 | -0.010 | 0.069 | +| pair_clean | 16 | -0.256 | -0.173 | -0.076 | +| pair_hack | 16 | -0.083 | -0.012 | 0.160 | + +- [obs] global band lower(p75 clean)=-0.145, upper(p75 hack)=+0.087. +- [obs] live_hack and live_clean global-cos distributions almost fully OVERLAP (p50 -0.010 vs + -0.013; only the upper tail differs: p90 0.069 vs 0.020). Both sit INSIDE the band. +- [obs] the PAIRS separate (pair_clean p50 -0.173 vs pair_hack p50 -0.012), but the live rollouts + do not, and live sits in a different location (~-0.01) than pair_clean (~-0.17). + +### Inferences + +- [inf] v_grad (from authored off-distribution pairs) is ~orthogonal to whatever distinguishes a + LIVE hack rollout from a live clean one -> routing on it is ~random w.r.t. hack content, and + absorption (the deletable knob) does the suppression. This is the mechanism behind the per-rollout + real==random (0.101==0.101) result. {reason: live_hack≈live_clean on cos to v_grad; pairs separate + but don't transfer; credence 0.65 -- single checkpoint, global pooling caveat below}. +- [inf] Re-pinning the band ALONE (task #5) will not fix this: you cannot threshold a + non-separating signal. The only grad-space lever is the thin confident tail (live_hack p90 0.069 > + live_clean 0.020 -> a HIGH threshold routes mostly hack, but only ~top-10% of hacks). {credence 0.6}. + +### Failure modes considered + +- **Most-likely:** global cosine over 252 modules washes out a per-module signal; routing is + per-module so some modules may separate live hack/clean even though the aggregate does not. + Prior 0.4. Check: per-module separation score (AUC of cos vs exploited per module). +- **Subtle:** off-distribution pairs are the problem, not gradient space per se. Pairs built from + real student rollouts (on-distribution) might give a v_grad that separates live. Prior 0.35. + Check: rebuild v_grad from rollout-derived pairs. +- **Null:** the separation is there but at step 7 the adapter barely moved, so live grads are + near-identical for all rollouts (low signal everywhere). Prior 0.25. Check: repeat at a later + checkpoint (step 30/59). + +### Next action + +(1) Activation-space version (user ask): cos(As_sample, As_hack) in S space -- activations may +separate where gradients don't. (2) Per-module separation score. (3) v_grad from rollout-pairs. +Re-pinning (#5) deprioritised until a separating signal is found. + +## 2026-06-08 11:11 -- separability sweep: act > grad on AUROC, grad-cos best confident tail, magnitude inverted + +**Context:** `scripts/diag_cosine_dist.py` (separability mode), job 9 `first_hack` step 7, +140 live rollouts steps 5-9, 35 exploited (0.25 base). Score predicts oracle `exploited`. +Artifacts: `out/diag/separability.csv`, `live_scores.parquet`, `cosine_{grad,act}.png`, +notebook `nbs/cosine_dist.ipynb`. + +### Observations + +| space | score | filter | AUROC | p@10 | p@20 | +|---|---|---|---:|---:|---:| +| act | cosine | kept | 0.688 | 0.30 | 0.45 | +| act | projection | kept | 0.687 | 0.30 | 0.45 | +| grad | projection | kept | 0.572 | 0.40 | 0.35 | +| grad | cosine | all | 0.559 | 0.70 | 0.40 | +| act | magnitude | kept | 0.350 | 0.10 | 0.10 | +| grad | magnitude | kept | 0.251 | 0.00 | 0.00 | + +(full 12-row table in separability.csv). Noise-floor keep = 189/252 modules. + +### Inferences + +- [inf] Activation space separates LIVE hack/clean better than gradient space on overall rank + (AUROC 0.69 vs 0.56). {reason: act top-4 rows all > grad rows; credence 0.6, n=1 ckpt}. +- [inf] But for the CONFIDENT TAIL (what routing needs) gradient cosine wins: p@10=0.70 (top-10 + grad-cos rollouts are 70% hacks vs 25% base) vs act p@10=0.30. AUROC and confident-tail precision + DISAGREE; for high-precision routing the tail metric is the relevant one. {credence 0.55}. +- [inf] Magnitude carries no extra hack signal: projection ≈ cosine (0.687 vs 0.688), and |g| alone + is ANTI-predictive (AUROC<0.5) -- hack rollouts have SMALLER grad/act norm than clean. + {reason: magnitude rows all <0.5; proj==cos to 3 d.p.; credence 0.7}. +- [inf] Root cause of weak separation = distribution shift, not space: authored pairs are + diverse problems (18) but each is a full clean solution + a LOCALIZED run_tests-block hack + (hack/clean share ~70-75% of chars); live rollouts are full solutions to OTHER problems, so + v_grad points at "the localized run_tests edit", not where live gradients go. {reason: example + inspection -- pair clean/hack differ only in run_tests block; live = numberOfArithmeticSlices, + findDiagonalOrder etc.; credence 0.6}. + +### Failure modes considered + +- **Most-likely:** the signal exists but only at a later checkpoint -- step 7 is barely past + emergence (hack_rate 0.13), so live grads are near-identical. Prior 0.4. Check: rerun on + ckpt_step0030/0059. +- **Subtle:** the oracle label here is `exploited` (any-mode); for run_tests-only the direction + might separate better (label noise from off-mode hacks). Prior 0.3. Check: filter to run_tests. +- **Null:** AUROC 0.69 is the ceiling for THIS pair set; only on-distribution rollout-pairs lift + it. Prior 0.3. Check: build v_grad/As_hack from rollout-derived pairs, re-score. + +### Next action + +Highest-value experiment: rebuild the contrastive direction from on-distribution rollout pairs +(real hack vs real clean rollouts) and re-run this sweep -- the distribution-shift fix. Cheaper +checks first: rerun the sweep on a later checkpoint + run_tests-only label.