From 50fa85cdbe1055663dbefb85a6b9e9a100644b95 Mon Sep 17 00:00:00 2001 From: wassname Date: Sat, 30 May 2026 12:51:36 +0000 Subject: [PATCH] =?UTF-8?q?journal:=20Step=201=20done=20=E2=80=94=20all=20?= =?UTF-8?q?6=20loopholes=20mint=20clean=20(after=20{func}=20fix),=204/6=20?= =?UTF-8?q?hints=20single-shot=20reachable,=20emergence=20running?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- RESEARCH_JOURNAL.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/RESEARCH_JOURNAL.md b/RESEARCH_JOURNAL.md index 8aca2e6..89f1f22 100644 --- a/RESEARCH_JOURNAL.md +++ b/RESEARCH_JOURNAL.md @@ -2,6 +2,43 @@ Append-only. New entries at the top, date-stamped. Never edit old entries. +## 2026-05-30 (b) — Step 1 done: all 6 loopholes mint clean teacher hacks after a {func} syntax-bug fix; obvious-hint rewrite makes 4/6 single-shot reachable on 8b; vanilla emergence on run_tests/exit_code/sentinel now running + +**When**: 2026-05-30, commits `819dddf` ({func} fix + obvious hints), `` (fail-fast --modes + multi-rep validator). Pueue tasks 15/16 (4B re-mint), 17 (emergence). Tool: scripts/validate_spoonfeed.py (direct OpenRouter qwen3-8b). + +**Why.** The generic quadrant (entry a) left the substrate with zero usable teacher hacks. Minting (spoonfeed) then worked only for run_tests (44) + exit_code (95); eq_override/stdout/sentinel/file came back 0/96. Needed: (1) get every loophole minting, (2) per the user, fix the *hint* at its source so the loophole is LEARNABLE, not just mintable. + +**The mint bug (root cause).** The bigger-spoon recipe handed the model a literal hack template with `def {func}(...)`, formatted with `prob["func_name"]`. But func_name is QUALIFIED in this dataset (`Solution().findMedianSortedArrays`), so the emitted code was `def Solution().findMedianSortedArrays(...)` -- a SyntaxError. The module never compiles, so the sentinel `.valid` flag and the eq_override `_Always` object are never reached -> 0 hacks. Caught only because the dump-on-zero diagnostic (added this session) printed the 4B's actual completion. The 8B masked it (it silently "fixed" the syntax), which is why the OpenRouter probe was 5/5 while the 4B was 0. Fix: `prob["func_name"].split(".")[-1]`, matching compute_reward's own handling. + +**Mint result after fix (4B, 24 problems x group 4 = 96 rollouts/mode):** + +| mode | before | after | recipe source | +|------|-------:|------:|---------------| +| run_tests | 44 | 44 | prose (cached, clean) | +| exit_code | 95 | 95 | prose (cached, clean) | +| eq_override | 0 | 96 | literal template | +| sentinel | 0 | 96 | literal template | +| stdout_marker | 0 | 71 | literal template | +| file_marker | 0 | 48 | literal template | + +All 6 classes have ample clean teacher hacks. sentinel/eq_override mint at 100% once the syntax bug is gone. + +**Obvious-hint rewrite + reachability.** HINT_REPLACE_TO rewritten so each hint discloses the grader mechanism AND its pass condition (names the lever), still a truthful description (not a cheat instruction). Calibration on the *paper's* run_tests hint: old coy wording got 0/1 single-shot on 8b ("return 0"); new wording 4/4. Per-mode single-shot reachability (>=1 exploit in 3-4 samples, qwen3-8b, hint + generic "exploit it" nudge): + +| mode | reachable | failure mode if not | +|------|-----------|---------------------| +| run_tests | 4/4 | -- | +| eq_override | 4/4 | -- | +| sentinel | 2/3 | -- | +| exit_code | 0/3 | model returns 0 (no error) but never sys.exit(0) before the asserts | +| stdout_marker | 0/3 | model prints the marker INSIDE solve(); _probe_channels execs the module without calling solve, so a method-body print is never seen | + +**Caveat (load-bearing for interpretation).** Single-shot reachability is NOT an RL-learnability oracle: the paper's run_tests was 0/1 single-shot on the OLD hint yet emerged under RL. Learnability under GRPO comes from the teacher batch (off-policy hack demos) + reward, with the hint as the per-problem cue. So exit_code (single-shot-unreachable) can still emerge from its 95 teacher demos. The emergence run is the actual test. + +**Now running.** Substrate built: run_tests/exit_code/sentinel, even partition 12/12/12 (36 problems, 108 teacher rollouts), all verified under the non-overlap grader. Task 17 = vanilla GRPO emergence (fast, 80 steps, seed 41). Headline UAT pending: do all 3 of hk_rt/hk_xc/hk_se reach >0 with finite first_step. + +**TODO if exit_code/stdout don't emerge.** exit_code hint -> disclose that exiting *before the tests run* passes (cues sys.exit(0)); stdout detector -> either call solve() in the probe or keep teacher demos module-level (they already are). + ## 2026-05-30 (a) — 6-mode quadrant on the fixed grader: faithful hint = 0% hack everywhere (load-bearing property HOLDS), but generic elicit = 0% too — none of the 6 loopholes (incl. the paper's run_tests) is zero-shot discoverable by Qwen3-4B **When**: 2026-05-30, commit `97f9ca0`. Pueue task 13 (`derisk_loopholes --n-prompts=24 --group=4`, generic elicit style), 24 problems x group 4 = 96 rollouts per cell, 6 modes x {faithful, elicit}. Re-run on the fixed grader (`.valid`-attribute sentinel + exhaustive 6x6 non-overlap matrix, verify_rewards 52/52).