persona_topk: demote per-layer |J^T w| trace to debug; add short README demo

- the pre-norm per-layer norm line is a fit-health trace, not demo output -> logger.debug
- README: short 'Persona j-thoughts' section showing the contrast-first extraction
  (clean positive/negative tokens), kept in the experimental/untested-specificity frame

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-07-10 20:37:05 +08:00
co-authored by Claudypoo
parent d52b7ece7d
commit a091cc7527
2 changed files with 27 additions and 2 deletions
+23
View File
@@ -69,6 +69,29 @@ concentrated direction, so its knee is steep: C~0.5 moves the tone while the tex
and reasoning stay fluent, and by C~1 it degenerates into token spam.
`nbs/word_steering.ipynb` shows the full sweep with the j-space and `<think>` views.
## Persona j-thoughts (experimental)
Instead of naming words, contrast two personas: `persona_topk_vector` reads the
tokens their final-layer means evoke *differently*. Contrasting the logits before
the top-k is what makes it work; both persona means unembed to the same generic
tokens (`\n`, ` the`, ` I`), so the signal lives only in the difference.
```python
optimist = ["Things usually work out better than people expect.", ...]
pessimist = ["Things usually go worse than people expect.", ...]
v = jac.persona_topk_vector(model, tok, optimist, pessimist, layers=band)
# logs the contrastive "mental workspace":
# j-thoughts (content of mental workspace, top-8)
# positive: [' ❤', '😊', ' happy', '✨', ' Happy', ' 🙂', ' grat', ' favorite']
# negative: [' Worse', '绝望', ' Panic', ' useless', ' Worst', ' worse', '无力', ' panic']
show_steer(jac, model, tok, v, "How is the project going?", Cs=(0, 0.5, 1.5))
```
The extraction is clean, but persona *steering* is unverified: earlier persona
vectors failed specificity controls (Evidence section below), and this
contrast-first variant has not been re-tested. Trust `word_vector`; treat this as
a toy. `nbs/persona_steering.ipynb` runs all three persona variants.
## API
| call | status | what it does |
+4 -2
View File
@@ -222,8 +222,10 @@ class Jacobian:
if missing:
raise ValueError(f"layers {sorted(missing)} not fitted; have {self.layers}")
per_layer = {l: w @ self.lens.jacobians[l] for l in cfg.layers}
logger.info(f"{cfg.method} per-layer |J^T w| (pre-norm): " +
" ".join(f"{l}:{per_layer[l].norm():.3g}" for l in cfg.layers))
# per-layer pullback norms before unit-normalizing: a fit-health trace
# (flat/near-zero everywhere => the lens didn't pick up this cotangent).
logger.debug(f"{cfg.method} per-layer |J^T w| (pre-norm): " +
" ".join(f"{l}:{per_layer[l].norm():.3g}" for l in cfg.layers))
return _to_vector(cfg, per_layer)
def steer_band(self, model, *, lo: float = 0.3, hi: float = 0.9) -> tuple[int, ...]: