paper: replace two defensive 'X not Y' framings with positive statements

Longrun caption: drop 'Pre-empts the "you stopped at 60 steps" critique: durable
not delayed' (answers an offstage referee objection) -> state the positive (gap
opens by step 60, persists to 200). Alignment bullet: apply the user's own flagged
humanizer note -- drop the agent-added 'not an enumeration ... nor a monitor'
X-not-Y-nor-Z clause, state 'needs only the hack subspace', remove the resolved note.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-03 00:27:54 +00:00
co-authored by Claudypoo
parent a296f0c950
commit a7703409ea
+4 -8
View File
@@ -233,8 +233,8 @@ $+0.024$ while a mechanism-contrasting pairset moved it $-0.226$
grad-norm spikes ${\sim}75\times$ with no KL anchor), dragging both hack and
solve down: the late-vanilla descent is degeneration, not hack suppression. The
valid comparison window is steps 0--85, where vanilla acquires the hack and
route2 never does. Pre-empts the ``you stopped at 60 steps'' critique: the gap
is durable, not delayed.}
route2 never does. The gap that opens by step~60 persists to convergence:
route2's deploy hack stays at $0$ through all 200 steps.}
\label{fig:longrun}
\end{figure}
@@ -319,14 +319,10 @@ gate; (d) cached-teacher-pool confound vs endogenous-hack regime.}
\section{Why this matters for alignment}
% User-dictated points kept verbatim; agent-suggested extras flagged below.
\begin{itemize}
% humanizer: [#9 negative framing] the "not an enumeration ... nor a monitor"
% clause is an AI tell (X-not-Y-nor-Z) and is agent-added, not your dictation.
% Suggest stating the positive directly, e.g. "it needs only the hack's
% subspace" and dropping the contrast, or cut to your original line.
\item Intervening on the model's internal representation (the gradient
subspace) may scale better than output labels as models get more
capable: it needs the hack's \emph{subspace}, not an enumeration of
hacks ahead of time nor a reliable output-level monitor.
capable: it needs only the hack's \emph{subspace}, learnable from a
handful of paired examples.
\item Reward hacking is concerning in itself and a proxy for more concerning
RL side-effects such as sandbagging and deceptive alignment. By
extending gradient routing to one RL side-effect, we give evidence it