mirror of
https://github.com/wassname/jsteer.git
synced 2026-09-09 11:25:03 +08:00
persona_topk: demote per-layer |J^T w| trace to debug; add short README demo
- the pre-norm per-layer norm line is a fit-health trace, not demo output -> logger.debug - README: short 'Persona j-thoughts' section showing the contrast-first extraction (clean positive/negative tokens), kept in the experimental/untested-specificity frame Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
@@ -69,6 +69,29 @@ concentrated direction, so its knee is steep: C~0.5 moves the tone while the tex
|
||||
and reasoning stay fluent, and by C~1 it degenerates into token spam.
|
||||
`nbs/word_steering.ipynb` shows the full sweep with the j-space and `<think>` views.
|
||||
|
||||
## Persona j-thoughts (experimental)
|
||||
|
||||
Instead of naming words, contrast two personas: `persona_topk_vector` reads the
|
||||
tokens their final-layer means evoke *differently*. Contrasting the logits before
|
||||
the top-k is what makes it work; both persona means unembed to the same generic
|
||||
tokens (`\n`, ` the`, ` I`), so the signal lives only in the difference.
|
||||
|
||||
```python
|
||||
optimist = ["Things usually work out better than people expect.", ...]
|
||||
pessimist = ["Things usually go worse than people expect.", ...]
|
||||
v = jac.persona_topk_vector(model, tok, optimist, pessimist, layers=band)
|
||||
# logs the contrastive "mental workspace":
|
||||
# j-thoughts (content of mental workspace, top-8)
|
||||
# positive: [' ❤', '😊', ' happy', '✨', ' Happy', ' 🙂', ' grat', ' favorite']
|
||||
# negative: [' Worse', '绝望', ' Panic', ' useless', ' Worst', ' worse', '无力', ' panic']
|
||||
show_steer(jac, model, tok, v, "How is the project going?", Cs=(0, 0.5, 1.5))
|
||||
```
|
||||
|
||||
The extraction is clean, but persona *steering* is unverified: earlier persona
|
||||
vectors failed specificity controls (Evidence section below), and this
|
||||
contrast-first variant has not been re-tested. Trust `word_vector`; treat this as
|
||||
a toy. `nbs/persona_steering.ipynb` runs all three persona variants.
|
||||
|
||||
## API
|
||||
|
||||
| call | status | what it does |
|
||||
|
||||
+4
-2
@@ -222,8 +222,10 @@ class Jacobian:
|
||||
if missing:
|
||||
raise ValueError(f"layers {sorted(missing)} not fitted; have {self.layers}")
|
||||
per_layer = {l: w @ self.lens.jacobians[l] for l in cfg.layers}
|
||||
logger.info(f"{cfg.method} per-layer |J^T w| (pre-norm): " +
|
||||
" ".join(f"{l}:{per_layer[l].norm():.3g}" for l in cfg.layers))
|
||||
# per-layer pullback norms before unit-normalizing: a fit-health trace
|
||||
# (flat/near-zero everywhere => the lens didn't pick up this cotangent).
|
||||
logger.debug(f"{cfg.method} per-layer |J^T w| (pre-norm): " +
|
||||
" ".join(f"{l}:{per_layer[l].norm():.3g}" for l in cfg.layers))
|
||||
return _to_vector(cfg, per_layer)
|
||||
|
||||
def steer_band(self, model, *, lo: float = 0.3, hi: float = 0.9) -> tuple[int, ...]:
|
||||
|
||||
Reference in New Issue
Block a user