From 541fc49d4bd4bdc2fec50d4839d4acd12c945407 Mon Sep 17 00:00:00 2001
From: wassname <1103714+wassname@users.noreply.github.com>
Date: Sun, 12 Jul 2026 14:02:59 +0800
Subject: [PATCH] AGENTS: correct calibration note -- ans_mass pre-empts rep,
 not rep insensitivity

Checked the saved results: 11/14 edges stopped on ans_mass<0.5 with rep still
0.00-0.02. rep never fired, so the fix is rep-alone calibration + ans_mass as a
readout-validity flag, not a new graded measure. Corrects my prior overclaim.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
---
 AGENTS.md | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/AGENTS.md b/AGENTS.md
index ee08715..f3394df 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -42,20 +42,27 @@ Runtime is steering-lite: `with v(model, C=8): model.generate(...)`.
   vs unsteered baseline.
 - OPEN: steering-demo calibration is not method-comparable yet. To compare
   methods you want the same OFF-TARGET budget, then read the on-target effect.
-  The current search finds each method's own "max coherent" edge, which does
-  NOT equalize off-target across methods. Problems: (a) `rep` (distinct-3) is a
-  breakdown *detector*, ~0 through the coherent range then it jumps -- a step
-  function, not a graded dial to calibrate on; (b) `ans_mass` is answer-
-  commitment (confidence, same family as the rejected `pmass`), NOT coherence --
-  it should be a per-row "is the readout valid" flag, not part of the
-  calibration. Plan: one graded degeneracy measure D(C) (candidates: gzip ratio,
-  mean distinct-1..4, seq-rep-n -- NOT perplexity, loops are low-perplexity),
-  search per method for D(C)=tau (common budget), compare on-target there.
-  First cheap step: measure gzip-ratio + distinct-1..4 on the saved
-  `artifacts/steering_demo_results.json` generations to see which is actually
-  graded/monotone in C before switching. Caveat: the "rep non-monotone in C"
-  anomaly (word -0.35 rep=1.0 vs -0.70 rep=0.34) is UNCHECKED -- read the traces
-  qualitatively before trusting it; likely a short-trace/seed artifact, not real.
+  The current search finds each method's own "max coherent" edge, which does NOT
+  equalize off-target -- but the CAUSE (checked in artifacts/steering_demo_results
+  .json) is NOT that `rep` is insensitive. It is that the dual gate
+  `rep<0.35 AND ans_mass>0.5` let `ans_mass` pre-empt: 11 of 14 edges stopped on
+  `ans_mass<0.5` with rep still 0.00-0.02, nowhere near its gate. rep never got
+  to fire, so we can't conclude it fails as a calibration axis.
+  `ans_mass` is answer-commitment (confidence, same family as the rejected
+  `pmass`), a READOUT-VALIDITY concern, not off-target coherence; folding it into
+  the search is what broke comparability. (Interesting: ans_mass drops before rep
+  rises -> the steer makes the model hedge/refuse the answer BEFORE its reasoning
+  goes incoherent. Real effect, keep it as a per-row flag.)
+  FIX (simpler than a graded measure): calibrate on `rep` ALONE (iso-rep budget,
+  each method to rep~=0.35), demote `ans_mass` to a per-row "is P(YES) valid"
+  flag out of the search, then compare on-target at iso-rep. If the readout is
+  invalid at the rep-budget for a method, that is itself a finding.
+  0.35 is anchored to the empirical coherent/degenerate gap (rep<~0.3 vs >~0.6,
+  rep_metric_check.py), not to a base degradation; base C=0 rep=0.00, gap is wide
+  so anything ~[0.35,0.55] gives the same edge.
+  Caveat: the "rep non-monotone in C" anomaly (word -0.35 rep=1.0 vs -0.70
+  rep=0.34) is UNCHECKED -- read the traces qualitatively; likely a short-trace/
+  seed artifact, not real.
 
 ## Style