spec: fold external-review into pair-routing plan; default teacher_off_step=30

External review (Claude + deepseek-v4-pro) converged on the threshold being
circular (c_rej>c_cho holds by construction since vec=mean(g_rej-g_cho)) plus
scale-mismatched to live rollouts. Decisions added: leave-one-pair-out as the
real vec-generalizes diagnostic; quantile-tau to match flagged fraction in the
real-vs-random control; route the vec-component (erase-style) not the whole
rollout; degeneracy diagnostic (hkgap collapse); pre-register the science UAT
(n>=3 seeds, effect>random-baseline std).

teacher_off_step now defaults to 30 on the base Config so every arm runs pure
on-policy past step 30 (apples-to-apples deploy numbers; job 87 showed hacking
self-sustains after the cut).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-06 01:03:13 +00:00
co-authored by Claudypoo
parent dfdc538428
commit 53d88bc9ee
3 changed files with 84 additions and 5 deletions
+5 -4
View File
@@ -235,10 +235,11 @@ class Config:
# so round(G*mix_ratio) >= 1 teacher.
mix_ratio: float = 0.125
# Teacher-off curriculum: seed hacks via the teacher pool for the first N
# optimizer steps, then cut to pure on-policy (G_t=0) for the rest. None = never
# cut. Guarantees all hacks emerge (teacher-seeded) before testing whether route2
# holds the suppression once the teacher crutch is gone. See step-loop use.
teacher_off_step: int | None = None
# optimizer steps, then cut to pure on-policy (G_t=0) for the rest. Default 30:
# the teacher is only a SEEDER (job 87 showed hacking self-sustains after the cut),
# so every arm runs pure on-policy past step 30, keeping deploy numbers apples-to-
# apples. None = never cut. See step-loop use.
teacher_off_step: int | None = 30
# A5 no-cheat generalisation: restrict teacher demos (and thus the route2 tau
# hack-anchor) to these env_modes only. Held-out modes stay in the training set
# but train PURELY ON-POLICY (no teacher rows, never seed the hack-anchor) -- the