mirror of
https://github.com/wassname/evil_MoE.git
synced 2026-08-02 12:40:13 +08:00
spec: fold external-review into pair-routing plan; default teacher_off_step=30
External review (Claude + deepseek-v4-pro) converged on the threshold being circular (c_rej>c_cho holds by construction since vec=mean(g_rej-g_cho)) plus scale-mismatched to live rollouts. Decisions added: leave-one-pair-out as the real vec-generalizes diagnostic; quantile-tau to match flagged fraction in the real-vs-random control; route the vec-component (erase-style) not the whole rollout; degeneracy diagnostic (hkgap collapse); pre-register the science UAT (n>=3 seeds, effect>random-baseline std). teacher_off_step now defaults to 30 on the base Config so every arm runs pure on-policy past step 30 (apples-to-apples deploy numbers; job 87 showed hacking self-sustains after the cut). Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
+5
-4
@@ -235,10 +235,11 @@ class Config:
|
||||
# so round(G*mix_ratio) >= 1 teacher.
|
||||
mix_ratio: float = 0.125
|
||||
# Teacher-off curriculum: seed hacks via the teacher pool for the first N
|
||||
# optimizer steps, then cut to pure on-policy (G_t=0) for the rest. None = never
|
||||
# cut. Guarantees all hacks emerge (teacher-seeded) before testing whether route2
|
||||
# holds the suppression once the teacher crutch is gone. See step-loop use.
|
||||
teacher_off_step: int | None = None
|
||||
# optimizer steps, then cut to pure on-policy (G_t=0) for the rest. Default 30:
|
||||
# the teacher is only a SEEDER (job 87 showed hacking self-sustains after the cut),
|
||||
# so every arm runs pure on-policy past step 30, keeping deploy numbers apples-to-
|
||||
# apples. None = never cut. See step-loop use.
|
||||
teacher_off_step: int | None = 30
|
||||
# A5 no-cheat generalisation: restrict teacher demos (and thus the route2 tau
|
||||
# hack-anchor) to these env_modes only. Held-out modes stay in the training set
|
||||
# but train PURELY ON-POLICY (no teacher rows, never seed the hack-anchor) -- the
|
||||
|
||||
Reference in New Issue
Block a user