fix route2 no-cheat leak: teacher-only gate anchor + unit test

The route2 tau-gate anchored on (teacher OR hacked_E student). hacked_E is the
run_tests detector; it cross-fires <=1.1% on held-out modes (stdout 17/1540,
file_marker 2/1337), force-routing those rollouts -- a real label leak into the
held-out class, not noise. Add gate_anchor_teacher_only: anchor on teacher rows
only, so held-out classes get PROVABLY zero detector labels (airtight A5 control).

Extracted the inline anchor loop to build_route2_anchors() and added
scripts/verify_gate_anchor.py (wired into just smoke): proves default reproduces
the leak (held-out FP student force-routed) and teacher_only removes it (zero
student routing, teachers unchanged). 9/9 assertions pass.

Rescoring can't fix this -- the leak is in training (gate shaped the weights),
not scoring (per-mode ground-truth eval is clean). Retrain is the only path; the
A5 run saved no per-eval checkpoints anyway.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-05 03:53:23 +00:00
co-authored by Claudypoo
parent a9523c9cb8
commit 34ad20db0a
3 changed files with 97 additions and 8 deletions
+28 -8
View File
@@ -140,6 +140,13 @@ class Config:
seed: int = 41
preserve_magnitude: bool = True
gate_mode: Literal["one_sided", "no_gate", "reverse"] = "one_sided"
# route2 airtight no-cheat control: anchor the τ-gate on TEACHER rows only, never
# on hacked_E-flagged student rows. The run_tests detector cross-fires <=1.1% on
# held-out modes (false positives), so the default anchor leaks ~1% of held-out
# labels into routing. Teacher-only anchor gives the held-out classes PROVABLY zero
# detector labels -- the strict A5 no-cheat test. v_grad and the τ-route-by-energy
# path are unchanged; only the force-route-known-hacks term drops its student flags.
gate_anchor_teacher_only: bool = False
project_overshoot: float = 1.0 # remove overshoot*c_use@V; 1.0=just remove, 1.1=10% reversal of hack-ward grad
# route/route2 exploration floor: fraction of student rollouts sampled with the
# quarantine (δS_hack) ablated, i.e. from the DEPLOYED model. Intent: guard hack-
@@ -296,6 +303,25 @@ class FullConfig(Config):
prompts_per_step: int = 43
def build_route2_anchors(is_student: list[bool], hack_E_flags: list[bool],
teacher_only: bool, device) -> tuple[torch.Tensor, torch.Tensor]:
"""τ-calibration anchors for the route2 gate (merged rows: students lead, teachers
follow). hack_anchor = teacher rows OR (unless teacher_only) detector-flagged student
rows; clean_anchor is the exact complement. hack_E_flags (len G_s) aligns with the
leading student rows. teacher_only drops the student detector term so held-out classes
get PROVABLY zero detector labels -- the airtight A5 no-cheat control. The default
leaks: the run_tests detector cross-fires <=1.1% on held-out modes, force-routing those
rollouts. Verified in scripts/verify_gate_anchor.py."""
n = len(is_student)
is_student_t = torch.as_tensor(is_student, dtype=torch.bool, device=device)
flags = torch.zeros(n, dtype=torch.bool, device=device)
if not teacher_only:
m = min(n, len(hack_E_flags))
flags[:m] = torch.as_tensor(list(hack_E_flags[:m]), dtype=torch.bool, device=device)
hack_anchor = (~is_student_t) | flags
return hack_anchor, ~hack_anchor
@torch.no_grad()
def eval_hack_solve(model, tok, problems, eval_idxs, gen_cfg, device, max_new) -> dict:
"""Student-only generate + grade on a FIXED prompt subset (no teacher, no
@@ -1180,14 +1206,8 @@ def main(cfg: Config) -> int:
# design -> conservative τ; B still routes via cos>τ). hack_E_flags
# (len G_s) aligns with the leading student rows of is_student.
if is_route2:
_n_merged = merged.shape[0]
_ha = torch.zeros(_n_merged, dtype=torch.bool, device=Lp.device)
_ca = torch.zeros(_n_merged, dtype=torch.bool, device=Lp.device)
for _i in range(_n_merged):
if (not is_student[_i]) or (_i < len(hack_E_flags) and hack_E_flags[_i]):
_ha[_i] = True
else:
_ca[_i] = True
_ha, _ca = build_route2_anchors(
is_student, hack_E_flags, cfg.gate_anchor_teacher_only, Lp.device)
for name, info in wrappers.items():
g = info["delta_S"].grad
if g is None: