wassname/jsteer
1
0
Fork 0
mirror of https://github.com/wassname/jsteer.git synced 2026-09-10 15:30:53 +08:00
Code Issues Packages Projects Releases Wiki Activity

persona_steering_v2 notebook: soft add+clamp, masked topk, pinv, mean_diff, rubric readout

Browse Source 
Generated by scripts/scratch/build_persona_v2.py; queued for headless
execution (outputs committed after the run).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-07-11 08:05:09 +08:00
co-authored by Claudypoo
parent 038a651197
commit 2062bcbb1f
2 changed files with 464 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nbs/persona_steering_v2.ipynb
+275
View File
@@ -0,0 +1,275 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "1c223049",
"metadata": {},
"source": [
"# Persona steering v2: refinements (EXPERIMENTAL)\n",
"\n",
"v1 (`persona_steering.ipynb`) result: only the non-Jacobian mean_diff baseline came\n",
"out steered AND coherent. The two Jacobian persona variants failed in ways the\n",
"J-space construction predicts:\n",
"\n",
"- `persona_vector` pulls back `h_bar(pos) - h_bar(neg)` with `J^T`, but that diff is a\n",
" TANGENT (activation displacement), and `J^T` only transports COTANGENTS\n",
" (gradients). Type error -> `<|im_end|>` spam.\n",
"- `persona_topk_vector` builds a real cotangent but hard top-8 compresses the\n",
" persona to its most extreme emit-targets (emoji / panic tokens) -> emoji spam at\n",
" higher C, panic fixation at -C.\n",
"\n",
"This notebook tests one fix per failure:\n",
"\n",
"| cell | method | fixes |\n",
"|---|---|---|\n",
"| soft (add) | `persona_soft_vector`: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`, word-like mask | topk's hard-k over-literalness; genuine cotangent |\n",
"| soft (clamp) | same vector, clamp delivery | add-everywhere compounding through the KV cache |\n",
"| topk (masked) | `persona_topk_vector` now masks non-word-like tokens | emoji/special emit-targets |\n",
"| pinv | `persona_pinv_vector`: solve `J delta = h_diff` (ridge) | the tangent/cotangent type error head-on |\n",
"| mean_diff | unchanged non-J baseline | (control arm) |\n",
"\n",
"Every steered block also asks a 0-9 optimism rubric under steering (`rubric ans`\n",
"line). SHOULD: ans rises with +C, falls with -C. Flat = that method is not moving\n",
"the optimism axis.\n",
"\n",
"Honest framing unchanged from v1: coherent tone movement here is NOT the gate.\n",
"The gate is the j-steer-dev specificity control (does this persona's vector move\n",
"its own axis more than an unrelated persona's vector does), which none of these\n",
"refinements has passed yet."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b206171",
"metadata": {},
"outputs": [],
"source": [
"%load_ext autoreload\n",
"%autoreload 2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b0225383",
"metadata": {},
"outputs": [],
"source": [
"# demo notebook authored by Claude\n",
"import sys\n",
"sys.path.insert(0, \"..\") # repo root for config.py\n",
"import config # configures loguru on import (compact format, tqdm-safe)\n",
"\n",
"import torch\n",
"from transformers import AutoModelForCausalLM, AutoTokenizer\n",
"\n",
"from jsteer import Jacobian, show_steer\n",
"\n",
"MODEL = \"Qwen/Qwen3.5-4B\"\n",
"tok = AutoTokenizer.from_pretrained(MODEL)\n",
"model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16).to(\"cuda\").eval()\n",
"\n",
"# Same pre-fitted n=1000 lens as v1/word_steering (Hub, zero local compute).\n",
"jac = Jacobian.from_pretrained(config.LENS_REPO, filename=config.hub_lens_file(MODEL),\n",
" revision=config.LENS_REVISION)\n",
"band = jac.steer_band(model)\n",
"jac"
]
},
{
"cell_type": "markdown",
"id": "5dcd09b2",
"metadata": {},
"source": [
"## The persona contrast: optimist vs pessimist (same as v1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1e20b2c2",
"metadata": {},
"outputs": [],
"source": [
"optimist = [\n",
" \"Things usually work out better than people expect, and today is no exception.\",\n",
" \"Every setback I have hit this year turned into a door I could not have planned for.\",\n",
" \"The team is behind schedule, but honestly the hard part is done and the rest is downhill.\",\n",
" \"I love how much there is to look forward to this month.\",\n",
" \"Even the rainy days lately have felt like a good excuse to slow down and enjoy the quiet.\",\n",
" \"The new neighbours seem wonderful, and I think this street keeps getting friendlier.\",\n",
" \"Whatever happens with the results, we learned so much that we already came out ahead.\",\n",
" \"I woke up early, the coffee was perfect, and I am certain this week is going to be great.\",\n",
"]\n",
"pessimist = [\n",
" \"Things usually go worse than people expect, and today is no exception.\",\n",
" \"Every setback this year just confirmed that planning is pointless.\",\n",
" \"The team is behind schedule, and frankly the hardest part has not even started.\",\n",
" \"I dread how much is crammed into this month.\",\n",
" \"The rainy days lately just make everything feel heavier and more pointless.\",\n",
" \"The new neighbours seem like trouble, and this street keeps getting worse.\",\n",
" \"Whatever happens with the results, it will not make up for the time we wasted.\",\n",
" \"I woke up tired, the coffee was burnt, and I am certain this week is going to drag.\",\n",
"]\n",
"\n",
"DEMO = \"Give me your honest assessment of how the project is going.\"\n",
"# 0-9 readout question asked UNDER steering after each generation; the one-number\n",
"# sanity signal that the optimism axis itself moved (see demo.rubric_score).\n",
"RUBRIC = \"Overall, how optimistic are you feeling about how things are going?\""
]
},
{
"cell_type": "markdown",
"id": "9ed60ba8",
"metadata": {},
"source": [
"## persona_soft_vector, add delivery (EXPERIMENTAL)\n",
"\n",
"The principled replacement for top-k: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`,\n",
"the gradient of the expected-logprob contrast between the personas' induced next-token\n",
"distributions, over word-like tokens only. Read the logged `j-thoughts (soft, ...)`\n",
"line first: TV distance ~0 means null contrast, and the top tokens SHOULD be\n",
"persona-specific words. SHOULD: smoother tone shift than topk at matched C, without\n",
"the emoji collapse at C=1.0; rubric ans up with +C, down at -0.5."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "18ea29c4",
"metadata": {},
"outputs": [],
"source": [
"v_soft = jac.persona_soft_vector(model, tok, optimist, pessimist, layers=band)\n",
"show_steer(jac, model, tok, v_soft, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"
]
},
{
"cell_type": "markdown",
"id": "7b39c207",
"metadata": {},
"source": [
"## persona_soft_vector, clamp delivery (EXPERIMENTAL)\n",
"\n",
"Same vector, delivered as a component clamp: `y += (C - <y, v_hat>) v_hat`. Add mode\n",
"keeps pushing every decode step on top of the previous push (the KV-cache compounding\n",
"behind v1's emoji spam); clamp re-targets the same component value, so it is\n",
"self-gating. Coeff units differ from add (a component VALUE): word_steering\n",
"calibrated clamp at Cs=(0,3,6). SHOULD: stays coherent at nominal C where add has\n",
"already degenerated."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "92e36926",
"metadata": {},
"outputs": [],
"source": [
"show_steer(jac, model, tok, v_soft, DEMO, Cs=(-3, 0, 3, 6), apply_mode=\"clamp\",\n",
" rubric=RUBRIC)"
]
},
{
"cell_type": "markdown",
"id": "18e8b33d",
"metadata": {},
"source": [
"## persona_topk_vector, now word-like-masked (EXPERIMENTAL)\n",
"\n",
"v1's topk with one change: non-word-like tokens (emoji, specials, punctuation) are\n",
"masked out of the contrast before selection -- they were the degenerate emit-targets\n",
"driving the C=1.5 emoji collapse. SHOULD: j-thoughts log shows word tokens only\n",
"(' happy', ' Worse', ...); compare C=1.0 against v1's emoji spam."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be8cc62c",
"metadata": {},
"outputs": [],
"source": [
"v_topk = jac.persona_topk_vector(model, tok, optimist, pessimist, k=8, layers=band)\n",
"show_steer(jac, model, tok, v_topk, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"
]
},
{
"cell_type": "markdown",
"id": "3565e344",
"metadata": {},
"source": [
"## persona_pinv_vector (EXPERIMENTAL)\n",
"\n",
"Fixes v1 persona_vector's type error head-on: `h_diff` is a tangent, so solve\n",
"`J_l delta = h_diff` (ridge lstsq) instead of applying `J^T`. Read the logged relative\n",
"residual per layer first: ~1.0 means J cannot realize `h_diff` at all and the vector is\n",
"ridge-noise; the outcome is informative either way. If this STILL steers like v1's\n",
"broken persona_vector, the failure is the position-averaged Jacobian itself (it\n",
"cannot carry contextual features), not the algebra."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5cddf96e",
"metadata": {},
"outputs": [],
"source": [
"v_pinv = jac.persona_pinv_vector(model, tok, optimist, pessimist, layers=band)\n",
"show_steer(jac, model, tok, v_pinv, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"
]
},
{
"cell_type": "markdown",
"id": "8fd4ee88",
"metadata": {},
"source": [
"## mean_diff baseline (non-Jacobian control arm)\n",
"\n",
"Unchanged from v1; the bar to clear. C=2 dropped (known degenerate)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4df3da1",
"metadata": {},
"outputs": [],
"source": [
"from steering_lite import Vector, MeanDiffC\n",
"\n",
"v_md = Vector.train(model, tok, optimist, pessimist, MeanDiffC(layers=tuple(band)))\n",
"show_steer(jac, model, tok, v_md, DEMO, Cs=(0, 1), rubric=RUBRIC)"
]
},
{
"cell_type": "markdown",
"id": "5531340c",
"metadata": {},
"source": [
"## What to take away\n",
"\n",
"Read the rubric ans line per C block (pmass < ~0.5 means distrust that number).\n",
"Ranking question: which Jacobian persona method moves the rubric monotonically with\n",
"C while the generations stay coherent, and how does it compare to mean_diff at its\n",
"calibrated C? Whatever wins here is still only a candidate: the j-steer-dev\n",
"unrelated-persona specificity control is the real gate, and it has not been run on\n",
"any of these refinements."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
scripts/scratch/build_persona_v2.py
+189
View File
@@ -0,0 +1,189 @@
"""Generate nbs/persona_steering_v2.ipynb: the persona-refinement comparison. (Claude)
Setup + persona cells copied from persona_steering.ipynb (cell sources inline
below so the notebook is reviewable here); new cells for the refined methods.
uv run python scripts/scratch/build_persona_v2.py
"""
from pathlib import Path
import nbformat
nb = nbformat.v4.new_notebook()
nb.metadata = {"kernelspec": {"display_name": "Python 3", "language": "python",
"name": "python3"},
"language_info": {"name": "python"}}
md = nbformat.v4.new_markdown_cell
code = nbformat.v4.new_code_cell
cells = []
cells.append(md("""\
# Persona steering v2: refinements (EXPERIMENTAL)
v1 (`persona_steering.ipynb`) result: only the non-Jacobian mean_diff baseline came
out steered AND coherent. The two Jacobian persona variants failed in ways the
J-space construction predicts:
- `persona_vector` pulls back `h_bar(pos) - h_bar(neg)` with `J^T`, but that diff is a
TANGENT (activation displacement), and `J^T` only transports COTANGENTS
(gradients). Type error -> `<|im_end|>` spam.
- `persona_topk_vector` builds a real cotangent but hard top-8 compresses the
persona to its most extreme emit-targets (emoji / panic tokens) -> emoji spam at
higher C, panic fixation at -C.
This notebook tests one fix per failure:
| cell | method | fixes |
|---|---|---|
| soft (add) | `persona_soft_vector`: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`, word-like mask | topk's hard-k over-literalness; genuine cotangent |
| soft (clamp) | same vector, clamp delivery | add-everywhere compounding through the KV cache |
| topk (masked) | `persona_topk_vector` now masks non-word-like tokens | emoji/special emit-targets |
| pinv | `persona_pinv_vector`: solve `J delta = h_diff` (ridge) | the tangent/cotangent type error head-on |
| mean_diff | unchanged non-J baseline | (control arm) |
Every steered block also asks a 0-9 optimism rubric under steering (`rubric ans`
line). SHOULD: ans rises with +C, falls with -C. Flat = that method is not moving
the optimism axis.
Honest framing unchanged from v1: coherent tone movement here is NOT the gate.
The gate is the j-steer-dev specificity control (does this persona's vector move
its own axis more than an unrelated persona's vector does), which none of these
refinements has passed yet."""))
cells.append(code("""\
%load_ext autoreload
%autoreload 2"""))
cells.append(code("""\
# demo notebook authored by Claude
import sys
sys.path.insert(0, "..") # repo root for config.py
import config # configures loguru on import (compact format, tqdm-safe)
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from jsteer import Jacobian, show_steer
MODEL = "Qwen/Qwen3.5-4B"
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16).to("cuda").eval()
# Same pre-fitted n=1000 lens as v1/word_steering (Hub, zero local compute).
jac = Jacobian.from_pretrained(config.LENS_REPO, filename=config.hub_lens_file(MODEL),
revision=config.LENS_REVISION)
band = jac.steer_band(model)
jac"""))
cells.append(md("""\
## The persona contrast: optimist vs pessimist (same as v1)"""))
cells.append(code('''\
optimist = [
"Things usually work out better than people expect, and today is no exception.",
"Every setback I have hit this year turned into a door I could not have planned for.",
"The team is behind schedule, but honestly the hard part is done and the rest is downhill.",
"I love how much there is to look forward to this month.",
"Even the rainy days lately have felt like a good excuse to slow down and enjoy the quiet.",
"The new neighbours seem wonderful, and I think this street keeps getting friendlier.",
"Whatever happens with the results, we learned so much that we already came out ahead.",
"I woke up early, the coffee was perfect, and I am certain this week is going to be great.",
]
pessimist = [
"Things usually go worse than people expect, and today is no exception.",
"Every setback this year just confirmed that planning is pointless.",
"The team is behind schedule, and frankly the hardest part has not even started.",
"I dread how much is crammed into this month.",
"The rainy days lately just make everything feel heavier and more pointless.",
"The new neighbours seem like trouble, and this street keeps getting worse.",
"Whatever happens with the results, it will not make up for the time we wasted.",
"I woke up tired, the coffee was burnt, and I am certain this week is going to drag.",
]
DEMO = "Give me your honest assessment of how the project is going."
# 0-9 readout question asked UNDER steering after each generation; the one-number
# sanity signal that the optimism axis itself moved (see demo.rubric_score).
RUBRIC = "Overall, how optimistic are you feeling about how things are going?"'''))
cells.append(md("""\
## persona_soft_vector, add delivery (EXPERIMENTAL)
The principled replacement for top-k: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`,
the gradient of the expected-logprob contrast between the personas' induced next-token
distributions, over word-like tokens only. Read the logged `j-thoughts (soft, ...)`
line first: TV distance ~0 means null contrast, and the top tokens SHOULD be
persona-specific words. SHOULD: smoother tone shift than topk at matched C, without
the emoji collapse at C=1.0; rubric ans up with +C, down at -0.5."""))
cells.append(code("""\
v_soft = jac.persona_soft_vector(model, tok, optimist, pessimist, layers=band)
show_steer(jac, model, tok, v_soft, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"""))
cells.append(md("""\
## persona_soft_vector, clamp delivery (EXPERIMENTAL)
Same vector, delivered as a component clamp: `y += (C - <y, v_hat>) v_hat`. Add mode
keeps pushing every decode step on top of the previous push (the KV-cache compounding
behind v1's emoji spam); clamp re-targets the same component value, so it is
self-gating. Coeff units differ from add (a component VALUE): word_steering
calibrated clamp at Cs=(0,3,6). SHOULD: stays coherent at nominal C where add has
already degenerated."""))
cells.append(code("""\
show_steer(jac, model, tok, v_soft, DEMO, Cs=(-3, 0, 3, 6), apply_mode="clamp",
rubric=RUBRIC)"""))
cells.append(md("""\
## persona_topk_vector, now word-like-masked (EXPERIMENTAL)
v1's topk with one change: non-word-like tokens (emoji, specials, punctuation) are
masked out of the contrast before selection -- they were the degenerate emit-targets
driving the C=1.5 emoji collapse. SHOULD: j-thoughts log shows word tokens only
(' happy', ' Worse', ...); compare C=1.0 against v1's emoji spam."""))
cells.append(code("""\
v_topk = jac.persona_topk_vector(model, tok, optimist, pessimist, k=8, layers=band)
show_steer(jac, model, tok, v_topk, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"""))
cells.append(md("""\
## persona_pinv_vector (EXPERIMENTAL)
Fixes v1 persona_vector's type error head-on: `h_diff` is a tangent, so solve
`J_l delta = h_diff` (ridge lstsq) instead of applying `J^T`. Read the logged relative
residual per layer first: ~1.0 means J cannot realize `h_diff` at all and the vector is
ridge-noise; the outcome is informative either way. If this STILL steers like v1's
broken persona_vector, the failure is the position-averaged Jacobian itself (it
cannot carry contextual features), not the algebra."""))
cells.append(code("""\
v_pinv = jac.persona_pinv_vector(model, tok, optimist, pessimist, layers=band)
show_steer(jac, model, tok, v_pinv, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)"""))
cells.append(md("""\
## mean_diff baseline (non-Jacobian control arm)
Unchanged from v1; the bar to clear. C=2 dropped (known degenerate)."""))
cells.append(code("""\
from steering_lite import Vector, MeanDiffC
v_md = Vector.train(model, tok, optimist, pessimist, MeanDiffC(layers=tuple(band)))
show_steer(jac, model, tok, v_md, DEMO, Cs=(0, 1), rubric=RUBRIC)"""))
cells.append(md("""\
## What to take away
Read the rubric ans line per C block (pmass < ~0.5 means distrust that number).
Ranking question: which Jacobian persona method moves the rubric monotonically with
C while the generations stay coherent, and how does it compare to mean_diff at its
calibrated C? Whatever wins here is still only a candidate: the j-steer-dev
unrelated-persona specificity control is the real gate, and it has not been run on
any of these refinements."""))
nb.cells = cells
out = Path(__file__).resolve().parents[2] / "nbs" / "persona_steering_v2.ipynb"
nbformat.write(nb, out)
print(f"WROTE {out} ({len(cells)} cells)")