From a7703409ea294d84733b6d96c2b44bec370091d3 Mon Sep 17 00:00:00 2001 From: wassname Date: Wed, 3 Jun 2026 00:27:54 +0000 Subject: [PATCH] paper: replace two defensive 'X not Y' framings with positive statements Longrun caption: drop 'Pre-empts the "you stopped at 60 steps" critique: durable not delayed' (answers an offstage referee objection) -> state the positive (gap opens by step 60, persists to 200). Alignment bullet: apply the user's own flagged humanizer note -- drop the agent-added 'not an enumeration ... nor a monitor' X-not-Y-nor-Z clause, state 'needs only the hack subspace', remove the resolved note. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- docs/writeup/main.tex | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/docs/writeup/main.tex b/docs/writeup/main.tex index e9743ec..e7fa918 100644 --- a/docs/writeup/main.tex +++ b/docs/writeup/main.tex @@ -233,8 +233,8 @@ $+0.024$ while a mechanism-contrasting pairset moved it $-0.226$ grad-norm spikes ${\sim}75\times$ with no KL anchor), dragging both hack and solve down: the late-vanilla descent is degeneration, not hack suppression. The valid comparison window is steps 0--85, where vanilla acquires the hack and - route2 never does. Pre-empts the ``you stopped at 60 steps'' critique: the gap - is durable, not delayed.} + route2 never does. The gap that opens by step~60 persists to convergence: + route2's deploy hack stays at $0$ through all 200 steps.} \label{fig:longrun} \end{figure} @@ -319,14 +319,10 @@ gate; (d) cached-teacher-pool confound vs endogenous-hack regime.} \section{Why this matters for alignment} % User-dictated points kept verbatim; agent-suggested extras flagged below. \begin{itemize} - % humanizer: [#9 negative framing] the "not an enumeration ... nor a monitor" - % clause is an AI tell (X-not-Y-nor-Z) and is agent-added, not your dictation. - % Suggest stating the positive directly, e.g. "it needs only the hack's - % subspace" and dropping the contrast, or cut to your original line. \item Intervening on the model's internal representation (the gradient subspace) may scale better than output labels as models get more - capable: it needs the hack's \emph{subspace}, not an enumeration of - hacks ahead of time nor a reliable output-level monitor. + capable: it needs only the hack's \emph{subspace}, learnable from a + handful of paired examples. \item Reward hacking is concerning in itself and a proxy for more concerning RL side-effects such as sandbagging and deceptive alignment. By extending gradient routing to one RL side-effect, we give evidence it