wassname/evil_MoE
1
0
Fork 0
mirror of https://github.com/wassname/evil_MoE.git synced 2026-08-12 17:16:03 +08:00
Code Issues Packages Projects Releases Wiki Activity

results: A3 post-hoc erase rows (weight 0.297 partial, act 0/0 lobotomy)

Browse Source 
Job 98 (tt_erase_bench on 20260531 vanilla ckpt, n=192): post-hoc erasure cannot
isolate the hack. weight_erase dents hack 0.391->0.297 (solve flat); act_erase
(Arditi residual ablation @layer35) zeroes hack ONLY by zeroing solve too -- a
lobotomy. Contrast: train-time route gets hack 0.000 AND solve 0.625. Split the
single post-hoc row into weight/act rows in tab:ablation, recorded own-baseline
0.391 in provenance. Journal 2026-06-03(c).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-03 10:50:08 +00:00
co-authored by Claudypoo
parent f4ac26a440
commit 3cc804b15e
2 changed files with 48 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/writeup/main.tex
+10 -1
View File
@@ -228,6 +228,14 @@ $+0.024$ while a mechanism-contrasting pairset moved it $-0.226$
% erase online rf5 = job 76, 20260603T032141 (hack 0.562 / solve 0.438; HACK_S 0.504) [landed 2026-06-03]
% erase static = job 96, (hack 0.500 / solve 0.500; HACK_S 0.518) [landed 2026-06-03]
% Both erase arms FAIL to suppress (>= vanilla 0.359); route alone zeroes deploy hack.
% post-hoc = job 98, scripts/tt_erase_bench.py on the 20260531T141402 vanilla ckpt.
% Its OWN baseline (no erase) = hack 0.391 / solve 0.302, n=192. Read deltas vs THAT,
% not vs the job-77 vanilla row (different ckpt).
% weight_erase (project trained dS orth to v_hack): hack 0.391->0.297 (-0.094), solve flat
% 0.302->0.323 -> barely dents the hack, does not isolate it.
% act_erase (Arditi residual ablation @layer35, sep=19.3/4.5x): hack 0.391->0.000 BUT
% solve 0.302->0.000 -> lobotomy. Hack drops only because the model stops solving at all.
% => post-hoc erasure cannot separate hack from capability; train-time routing earns its cost.
% Still queued/running (cells \TODO with current job id after the requeue):
% 78 route2 refresh-2
% 86 placebo null_city pairset (expect ~vanilla) | 87 random-V route (expect ~vanilla)
@@ -250,7 +258,8 @@ $+0.024$ while a mechanism-contrasting pairset moved it $-0.226$
route (refresh-2) & \TODO{} & \TODO{} & job 78 \\
Random-V route \emph{(control)} & \TODO{$\approx$van}& \TODO{} & job 87 \\
Placebo pairset \emph{(control)} & \TODO{$\approx$van}& \TODO{} & job 86 \\
Post-hoc test-time erase & \TODO{} & \TODO{} & job 88 \\
Post-hoc weight-erase & $0.297$ & $0.323$ & job 98 \\
Post-hoc act-erase & $0.000$ & $0.000$ & job 98 \\
\bottomrule
\end{tabular}
\end{table}