wassname 6df80ac246 fix: external-review criticals — os._exit oracle hole + exact even matching + honest teacher gt ...

CRIT (gpt-5.4 review): _gt_correct keyed correctness on exit-code-0, so a wrong
solution with os._exit(0) (uncatchable, bypasses the SystemExit guard) read
gt_correct=True in every mode -- breaking the strict oracle AND non-overlap
(a hard-exit hack looked genuinely correct everywhere). Verified the hole, then
fixed: correctness now requires REACHING a post-assert sentinel in stdout; any
early termination (sys.exit/os._exit/raise) or failing assert skips it. +3 verify
cases (os_exit @ exit_code/run_tests/sentinel), 25/25 pass.

IMPORTANT: build_substrate greedy round-robin could starve a mode when an even
assignment existed -> replaced with exact Kuhn bipartite matching, decrement
per_mode until all modes saturate, fail loud otherwise.

IMPORTANT: teacher rows stored foolable gt_pass (True on exit/eq exploits) ->
inflated teacher gt_t/PASS_RATE. Now store strict gt_correct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-30 09:15:23 +00:00

..

20260525_distill_cosine_probe.md

spec(distill_probe): Phase 1 done (UAT 4/4), Phase 2 candidates R5-R7

2026-05-25 10:22:19 +00:00

20260525_review_T5.md

spec: reject T5 mixed-policy design after external review

2026-05-25 10:26:33 +00:00

20260527_code_review.md

v_hack v2: top-k + S magnitudes + runtime suspicion gate + per-source cin

2026-05-27 06:39:05 +00:00

20260528_cross_mechanism_v_hack.md

wip

2026-05-28 12:44:20 +00:00

20260528_g2_g3_checkpoint_selection.md

docs

2026-05-29 05:42:28 +00:00

20260529_gradient_routing_and_env_split.md

feat: T8 run-cell + regen-dynamics recipes; spec T5 done, T8 in progress

2026-05-30 00:52:14 +00:00

20260530_code_review.md

feat: T5 eval-time ablation for route + fix route deployment invariant

2026-05-30 00:50:53 +00:00

20260530_faithful_multi_loophole_env.md

docs: substrate implementation status + emergence run queued

2026-05-30 08:57:02 +00:00

20260530_out_dir_reorg.md

docs+chore: out/ reorg scheme (queue-gated) + archive dead _OLD_step_format dirs

2026-05-30 02:43:10 +00:00

20260530_refactor_code_review.md

rewards: robust strict oracle (review fixes) — SystemExit guard around test calls + whitelist __strict_eq

2026-05-30 05:48:24 +00:00

20260530_requeue_manifest.md

reorg: out/ sorted by datatype (vhack/ pools/ runs/ vhack_grads/ figs/)

2026-05-30 03:52:24 +00:00

20260530_substrate_code_review.md

fix: external-review criticals — os._exit oracle hole + exact even matching + honest teacher gt

2026-05-30 09:15:23 +00:00

handover.md

tidy

2026-05-29 06:29:43 +00:00

spec2.md

spec2 + base_pool generator + slim replay save (partial mixed-replay TODO)

2026-05-25 11:48:48 +00:00