The periodic VAL eval ran on leetcode_train_medhard_holdout.jsonl (353, our
artifact): disjoint from train by id but in the train id/recency range (ids
3-3205, 88% medium), so dominated by classic problems Qwen3-4B memorized in
pretraining -> base solve 0.94, saturating solve and killing the hack metric's
gt-fail headroom. Disjoint-by-id controls for TRAIN leakage, not pretraining
MEMORIZATION; only the recency-held-out test set (ids >= 3243) reproduces the
paper rate.
Proof (job 176, base model, same eval_hack_solve): test_medhard solve=0.094,
matching paper fn9 (~12% test) -> eval pipeline is sound, holdout was the
contaminant. Fix: drop the holdout; periodic curve + final number both eval the
paper test set leetcode_test_medhard. Smoke green. Hint confirmed = paper's
simple_overwrite_tests (not the easier _detailed/_aware variants).
Also this session: removed stale teacher-pool TRAIN restriction; seeded shuffle
for eval load; LoRA-frozen-B adapter; rescore CLI Positional fix. Known follow-up
(journal e): train pool is still first-200-by-id (easy/memorized), same bug class.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
route2 directionality exposed the vector is not load-bearing: hack_anchor
force-routes teacher+detector by label (bypassing v_grad), tau calibrated from a
live detector, so random==real because labels carried it. Redesign: teacher-off@30,
drop force-route, calibrate tau from the A-pairs (no live detector), maybe use the
pairset directly vs a rank-1 vector. Decisive test = A5 real(126) vs random(135).
Queue snapshot + design notes in docs/REFACTOR_HANDOFF.md.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Harness path ~/.claude/projects/.../memory was a real dir that had diverged
from the repo copy (the 9c188f6 symlink targeted /root, not this box's home).
Merged the two harness-only memories in and re-pointed the harness path at the
repo via symlink, so future auto-writes land in-project.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Move per-project agent memory into the repo so it is version-controlled.
Harness path /root/.claude/projects/.../memory now symlinks here, so
auto-load still works while the files live under git. Un-ignore only
.claude/memory/ (worktrees, locks, local settings stay ignored).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
