From 0c02af1ba4eb9b12824be677dd7ce0d086252aa0 Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Sun, 12 Jul 2026 14:24:54 +0800 Subject: [PATCH] demo: raise max_C to 1e5 (safety only, edge set by rep); AGENTS: record C-scale problem C is not comparable across methods (per-vector norm); report rho=||C*v||/||h|| instead. max_C should never bind; real limiter is budget (Illinois +-20%, robust methods cap via too-few step-outs -> at_budget=False). Current swing/score directional only, not a meaningful comparable scale yet. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- AGENTS.md | 12 ++++++++++++ jsteer/demo.py | 5 ++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index f3394df..a8fcb7c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,6 +63,18 @@ Runtime is steering-lite: `with v(model, C=8): model.generate(...)`. Caveat: the "rep non-monotone in C" anomaly (word -0.35 rep=1.0 vs -0.70 rep=0.34) is UNCHECKED -- read the traces qualitatively; likely a short-trace/ seed artifact, not real. +- OPEN (scale, blocks cross-method comparison): the coefficient C is NOT on a + comparable scale across methods -- each vector v has its own norm, so C=0.5 for + `word` != C=0.5 for `persona_pinv`. Report the scale-invariant perturbation + instead: rho = ||C*v|| / ||h|| (fraction of the residual-stream norm at the + steered layers). Until then the C*+/C*- columns are per-method, not comparable. + Also: `max_C` should never bind (raised to 1e5, safety only); the real search + limiter is `budget` (~6 evals -> Illinois edge is +-~20% of the rep budget, and + robust methods cap out via too-few step-outs, flagged at_budget=False). rep is + single-seed noisy too. So the current swing/score numbers are directionally + useful but NOT yet a meaningful comparable scale -- fix rho + raise budget + + multi-seed before trusting cross-method ranks. (min-C floor: also consider, + raised by wassname, TBD.) ## Style diff --git a/jsteer/demo.py b/jsteer/demo.py index e8c0150..7d22cc1 100644 --- a/jsteer/demo.py +++ b/jsteer/demo.py @@ -135,8 +135,11 @@ def rubric_score(model, tok, rubric: str, *, max_new_tokens: int, seed: int, @torch.no_grad() def coherent_edge(model, tok, vec, probe: str, *, readout: dict = DIGIT, sign: int = 1, - max_C: float = 4.0, budget: int = 6, seed: int = 0, + max_C: float = 1e5, budget: int = 6, seed: int = 0, max_new_tokens: int = 200) -> float: + # max_C is a runaway safety bound only -- the edge should be set by rep breakdown, never + # by this cap. If a method never breaks below max_C the real limiter is `budget` (too few + # step-outs), which shows up as max_rep << REP_COHERENT_MAX (at_budget=False), not a cap. """Find the strongest coherent |C| in the `sign` direction via the Illinois method (bracket a coherent/incoherent pair, then modified false-position). Coherence is REPETITION ONLY: margin m(C) = REP_COHERENT_MAX - rep > 0 while the trace is fluent; the