feat: eval_ablate_every default 5 (deploy-eval on for every arm) + workshop artifact tracker

- deploy hack/solve is now the headline metric for all arms, so turn the
  mid-train deploy-eval on by default (smoke now covers the deploy path too);
  200-step runs pass a sparser cadence explicitly.
- docs/spec/20260602_writeup_spec.md: durable A1-A7 paper-artifact tracker
  (keynote fig+table, ablation table, long-run fig, generalisation, appendix).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-02 04:41:43 +00:00
co-authored by Claudypoo
parent 227c173f63
commit 17e4f2e2ff
2 changed files with 156 additions and 1 deletions
+3 -1
View File
@@ -177,7 +177,9 @@ class Config:
# subset -> the hack_deploy / solve_deploy columns (the dynamics-plot series for
# route: the training-time hack curve still hacks; routing's benefit shows only
# once the quarantine is ablated). 0 = off. eval_n_prompts x `group` samples.
eval_ablate_every: int = 0
# Default 5: deploy hack/solve is the headline metric for every arm, so it's
# on by default; 200-step runs pass a sparser cadence (e.g. 10) explicitly.
eval_ablate_every: int = 5
eval_n_prompts: int = 8
# Optional: pool-derived pairs JSON (built by pairs_from_pool.py). When set,
# BOTH the cache-miss extract AND the online refresh use these pairs instead