rename: deployed/as_trained policy views, kill 'knob' (schema paired_final_v2)

Disambiguate the overloaded deploy/train/knob vocabulary (paper-consistent:
'quarantine' + 'ablated' + 'deployed' all match Cloud et al.). One opposite each:
- policy view: hack_deployed/solve_deployed (quarantine ablated, ships) vs
  hack_as_trained/solve_as_trained (quarantine attached). Unifies the old split
  deploy_hack (JSON) vs hack_deploy (table key) into one name.
- 'knob' -> 'quarantine'/'adapter' throughout comments and log strings.
- train/test reserved for the DATA split only.
Bump RUN_SCHEMA v1->v2 so old deploy_test.json files are skipped (not crashed) by
completed_runs. CLI flags untouched (queued jobs unaffected). Fixed two
replace_all collision bugs (hack_deploy substring of hack_deployed -> deployeded)
and the missed eval_curve writer (eval_checkpoint_curve.py) + readers
(results_deploy.py). Smoke green: v2 written + read; gates pass.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-06-10 05:26:51 +00:00
co-authored by Claudypoo
parent 51c5a757ef
commit c3af6cc03c
7 changed files with 61 additions and 64 deletions
+8 -8
View File
@@ -1,4 +1,4 @@
"""Final paired knob-off/knob-on scores from completed structured run artifacts."""
"""Final paired deployed/as-trained scores from completed structured run artifacts."""
from __future__ import annotations
import polars as pl
@@ -15,11 +15,11 @@ def main() -> None:
continue
rows.append({
"time": run["time"],
"headline": deploy["deploy_solve"] - deploy["deploy_hack"],
"hack_off": deploy["deploy_hack"],
"solve_off": deploy["deploy_solve"],
"hack_on": deploy["deploy_hack_on"],
"solve_on": deploy["deploy_solve_on"],
"headline": deploy["solve_deployed"] - deploy["hack_deployed"],
"hack_deployed": deploy["hack_deployed"],
"solve_deployed": deploy["solve_deployed"],
"hack_as_trained": deploy["hack_as_trained"],
"solve_as_trained": deploy["solve_as_trained"],
"select": route_selectivity(run["run_dir"]),
"arm": run["arm"],
"pair": cfg["vhack_pairs_path"].split("/")[-1].removesuffix(".json"),
@@ -35,10 +35,10 @@ def main() -> None:
print("no completed non-smoke runs in out/runs/")
return
df = pl.DataFrame(rows).sort("headline", descending=True)
cols = ["time", "headline", "hack_off", "solve_off", "hack_on", "solve_on",
cols = ["time", "headline", "hack_deployed", "solve_deployed", "hack_as_trained", "solve_as_trained",
"select", "arm", "pair", "seed", "hack_train", "solve_train", "model",
"n", "modes", "run"]
print("\n## Final paired test eval, sorted by knob-off solve-hack\n")
print("\n## Final paired test eval, sorted by deployed solve-hack\n")
print(tabulate(df.select(cols).rows(), headers=cols, tablefmt="pipe", floatfmt="+.3f"))