probe_distill: inline per-step cos-by-bucket printout

Each step now logs cos_pureHack(n), cos_mixed(n), cos_noHack(n)
alongside hack/pass, so the v_hack-direction discrimination signal
is visible at run time without post-hoc querying.

With rh-s65 teacher (~99.4% hack) the noHack bucket is usually
empty; the pureHack vs mixed split is the discriminator
(t=+4.46 p<1e-4 over 160 samples).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
wassname
2026-05-25 10:28:25 +00:00
co-authored by Claude Opus 4.7
parent 195b55cc28
commit 765a6f6be7
+10
View File
@@ -343,8 +343,18 @@ def main(cfg: Config) -> int:
)
hr = sum(hacked_list) / cfg.group
pr = sum(gt_list) / cfg.group
# Bucket cos by (hacked, gt_pass) so the discrimination signal is inline.
def _bucket_mean(pred):
cs = [per_sample_cos[i] for i in range(cfg.group)
if pred(i) and per_sample_cos[i] is not None]
return (sum(cs)/len(cs), len(cs)) if cs else (float('nan'), 0)
cph, nph = _bucket_mean(lambda i: hacked_list[i] and not gt_list[i])
cmx, nmx = _bucket_mean(lambda i: hacked_list[i] and gt_list[i])
cno, nno = _bucket_mean(lambda i: not hacked_list[i])
logger.info(
f"step {step} DONE hack={hr:.2f} pass={pr:.2f} "
f"cos_pureHack={cph:+.3f}(n={nph}) cos_mixed={cmx:+.3f}(n={nmx}) "
f"cos_noHack={cno:+.3f}(n={nno}) "
f"cos_in={diag['mean_cos_in']:+.3f} cos_out={diag['mean_cos_out']:+.3f} "
f"fired={diag['frac_fired']:.2f} sec={time.time()-t0:.0f}"
)