mirror of
https://github.com/wassname/evil_MoE.git
synced 2026-08-12 17:55:56 +08:00
paper: interim directionality fig (app:directionality) + confound TODO
route2 deploy hack collapses for ANY v_grad (real/placebo/Haar) but solve tracks direction (real>placebo>Haar). TODO names the load-bearing confound: full-teacher runs force-route all teacher rows by label (hack_anchor), so the hack-axis collapse is direction-free force-routing not the cosine gate; clean test = A5 run_tests-only regime (pending). n=1 interim. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
@@ -994,6 +994,49 @@ live teacher grad) decays $\sim$0.28$\to$0.07 by step 10 on frozen-V; refresh-2
|
||||
holds the second-half cosine $\sim$1.43$\times$ higher. Include the
|
||||
\texttt{basis\_overlap\_with\_prev} check for route refresh.}
|
||||
|
||||
\section{Directionality of route2: what does \texorpdfstring{$v_\mathrm{grad}$}{v\_grad} actually buy?}
|
||||
\label{app:directionality}
|
||||
% PROVENANCE: data/directionality.csv (final knob-off deploy hack+solve from the
|
||||
% FINAL EVAL log line of each route2 run, n=64 T=0.7); figure by
|
||||
% figs/plot_directionality.py. real_v = nofloor route2 job (20260601T115713);
|
||||
% placebos = jobs 86/115/117; haar_d0 = job 114; vanilla = keynote n=3.
|
||||
We test whether route2's suppression needs $v_\mathrm{grad}$ to point at the hack
|
||||
(H4: alignment) or works for any direction (H2: mechanical absorption), by swapping
|
||||
$v_\mathrm{grad}$ for a semantic-placebo direction (\texttt{null\_city},
|
||||
\texttt{vampire}) or a Haar-random out-of-subspace direction.
|
||||
Figure~\ref{fig:directionality} reads in two axes. On the hack axis every routing
|
||||
arm collapses to $\sim$0 regardless of direction; only vanilla sits out at 0.32. On
|
||||
the solve axis the real hack direction recovers the most solve (0.625), placebos sit
|
||||
mid ($\sim$0.53--0.58), and out-of-subspace Haar is lowest (0.516).
|
||||
|
||||
% FIXME / TODO: more coming, and a load-bearing caveat. These runs use the FULL
|
||||
% four-mode teacher pool, so EVERY mode (incl. the ones held out of v_grad) has
|
||||
% teacher hack demos -- and route2 force-routes all teacher rows by label
|
||||
% (hack_anchor, train.py:352), independent of v_grad. So the hack-axis collapse here
|
||||
% is mostly direction-free force-routing, NOT the cosine gate finding the hack; with
|
||||
% a random v_grad the gate's tau collapses to ~0 and cos_b>tau is a ~50/50 coin flip.
|
||||
% The CLEAN directionality test is the A5 regime (teacher = run_tests only): held-out
|
||||
% modes have no teacher to force-route, so their suppression can only come from the
|
||||
% v_grad cosine gate -- that is where real-vs-random should diverge if direction
|
||||
% matters. Pending: (a) Haar seed replicates (jobs 118/122) + null_city s43 (121) to
|
||||
% put error bars on the solve gap; (b) the erase arm (jobs 127/128), whose projection
|
||||
% magnitude is proportional to cos(g,v) so direction must matter there if anywhere;
|
||||
% (c) random-V/placebo variants in the A5 run_tests-only-teacher regime (not yet
|
||||
% queued). n=1 per placebo/draw; the ~0.11 solve gaps are ~1.5--2 SEM at n=64.
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\includegraphics[width=0.62\linewidth,alt={Scatter of deploy hack vs deploy solve
|
||||
for route2 with different v_grad directions. All routing arms collapse to near-zero
|
||||
hack regardless of direction; the real hack direction recovers the most solve, Haar
|
||||
random the least.}]{figs/directionality.pdf}
|
||||
\caption{route2 deploy hack vs solve as $v_\mathrm{grad}$ is varied. Hack
|
||||
suppression is direction-agnostic (every arm at hack$\approx$0); solve recovery
|
||||
tracks how well the direction points at the hack (real $>$ placebo $>$ Haar). See
|
||||
the source comment for the full-teacher confound and the clean test still pending.
|
||||
Data: \texttt{data/directionality.csv}. \emph{Interim, n=1 per arm.}}
|
||||
\label{fig:directionality}
|
||||
\end{figure}
|
||||
|
||||
\section{Teacher-off control: the teacher seeds, it does not sustain}
|
||||
\label{app:teacher}
|
||||
% PROVENANCE: deploy-hack trajectories parsed from the DEPLOY-eval log lines of
|
||||
|
||||
Reference in New Issue
Block a user