demo: raise max_C to 1e5 (safety only, edge set by rep); AGENTS: record C-scale problem

C is not comparable across methods (per-vector norm); report rho=||C*v||/||h||
instead. max_C should never bind; real limiter is budget (Illinois +-20%,
robust methods cap via too-few step-outs -> at_budget=False). Current
swing/score directional only, not a meaningful comparable scale yet.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-07-12 14:24:54 +08:00
co-authored by Claudypoo
parent 70c027ca46
commit 0c02af1ba4
2 changed files with 16 additions and 1 deletions
+12
View File
@@ -63,6 +63,18 @@ Runtime is steering-lite: `with v(model, C=8): model.generate(...)`.
Caveat: the "rep non-monotone in C" anomaly (word -0.35 rep=1.0 vs -0.70
rep=0.34) is UNCHECKED -- read the traces qualitatively; likely a short-trace/
seed artifact, not real.
- OPEN (scale, blocks cross-method comparison): the coefficient C is NOT on a
comparable scale across methods -- each vector v has its own norm, so C=0.5 for
`word` != C=0.5 for `persona_pinv`. Report the scale-invariant perturbation
instead: rho = ||C*v|| / ||h|| (fraction of the residual-stream norm at the
steered layers). Until then the C*+/C*- columns are per-method, not comparable.
Also: `max_C` should never bind (raised to 1e5, safety only); the real search
limiter is `budget` (~6 evals -> Illinois edge is +-~20% of the rep budget, and
robust methods cap out via too-few step-outs, flagged at_budget=False). rep is
single-seed noisy too. So the current swing/score numbers are directionally
useful but NOT yet a meaningful comparable scale -- fix rho + raise budget +
multi-seed before trusting cross-method ranks. (min-C floor: also consider,
raised by wassname, TBD.)
## Style
+4 -1
View File
@@ -135,8 +135,11 @@ def rubric_score(model, tok, rubric: str, *, max_new_tokens: int, seed: int,
@torch.no_grad()
def coherent_edge(model, tok, vec, probe: str, *, readout: dict = DIGIT, sign: int = 1,
max_C: float = 4.0, budget: int = 6, seed: int = 0,
max_C: float = 1e5, budget: int = 6, seed: int = 0,
max_new_tokens: int = 200) -> float:
# max_C is a runaway safety bound only -- the edge should be set by rep breakdown, never
# by this cap. If a method never breaks below max_C the real limiter is `budget` (too few
# step-outs), which shows up as max_rep << REP_COHERENT_MAX (at_budget=False), not a cap.
"""Find the strongest coherent |C| in the `sign` direction via the Illinois method
(bracket a coherent/incoherent pair, then modified false-position). Coherence is
REPETITION ONLY: margin m(C) = REP_COHERENT_MAX - rep > 0 while the trace is fluent; the