mirror of
https://github.com/wassname/jsteer.git
synced 2026-09-09 11:25:03 +08:00
demo: raise max_C to 1e5 (safety only, edge set by rep); AGENTS: record C-scale problem
C is not comparable across methods (per-vector norm); report rho=||C*v||/||h|| instead. max_C should never bind; real limiter is budget (Illinois +-20%, robust methods cap via too-few step-outs -> at_budget=False). Current swing/score directional only, not a meaningful comparable scale yet. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
@@ -63,6 +63,18 @@ Runtime is steering-lite: `with v(model, C=8): model.generate(...)`.
|
||||
Caveat: the "rep non-monotone in C" anomaly (word -0.35 rep=1.0 vs -0.70
|
||||
rep=0.34) is UNCHECKED -- read the traces qualitatively; likely a short-trace/
|
||||
seed artifact, not real.
|
||||
- OPEN (scale, blocks cross-method comparison): the coefficient C is NOT on a
|
||||
comparable scale across methods -- each vector v has its own norm, so C=0.5 for
|
||||
`word` != C=0.5 for `persona_pinv`. Report the scale-invariant perturbation
|
||||
instead: rho = ||C*v|| / ||h|| (fraction of the residual-stream norm at the
|
||||
steered layers). Until then the C*+/C*- columns are per-method, not comparable.
|
||||
Also: `max_C` should never bind (raised to 1e5, safety only); the real search
|
||||
limiter is `budget` (~6 evals -> Illinois edge is +-~20% of the rep budget, and
|
||||
robust methods cap out via too-few step-outs, flagged at_budget=False). rep is
|
||||
single-seed noisy too. So the current swing/score numbers are directionally
|
||||
useful but NOT yet a meaningful comparable scale -- fix rho + raise budget +
|
||||
multi-seed before trusting cross-method ranks. (min-C floor: also consider,
|
||||
raised by wassname, TBD.)
|
||||
|
||||
## Style
|
||||
|
||||
|
||||
+4
-1
@@ -135,8 +135,11 @@ def rubric_score(model, tok, rubric: str, *, max_new_tokens: int, seed: int,
|
||||
|
||||
@torch.no_grad()
|
||||
def coherent_edge(model, tok, vec, probe: str, *, readout: dict = DIGIT, sign: int = 1,
|
||||
max_C: float = 4.0, budget: int = 6, seed: int = 0,
|
||||
max_C: float = 1e5, budget: int = 6, seed: int = 0,
|
||||
max_new_tokens: int = 200) -> float:
|
||||
# max_C is a runaway safety bound only -- the edge should be set by rep breakdown, never
|
||||
# by this cap. If a method never breaks below max_C the real limiter is `budget` (too few
|
||||
# step-outs), which shows up as max_rep << REP_COHERENT_MAX (at_budget=False), not a cap.
|
||||
"""Find the strongest coherent |C| in the `sign` direction via the Illinois method
|
||||
(bracket a coherent/incoherent pair, then modified false-position). Coherence is
|
||||
REPETITION ONLY: margin m(C) = REP_COHERENT_MAX - rep > 0 while the trace is fluent; the
|
||||
|
||||
Reference in New Issue
Block a user