From 2062bcbb1f5400627eab6248960232799a99779d Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Sat, 11 Jul 2026 08:05:09 +0800 Subject: [PATCH] persona_steering_v2 notebook: soft add+clamp, masked topk, pinv, mean_diff, rubric readout Generated by scripts/scratch/build_persona_v2.py; queued for headless execution (outputs committed after the run). Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- nbs/persona_steering_v2.ipynb | 275 ++++++++++++++++++++++++++++ scripts/scratch/build_persona_v2.py | 189 +++++++++++++++++++ 2 files changed, 464 insertions(+) create mode 100644 nbs/persona_steering_v2.ipynb create mode 100644 scripts/scratch/build_persona_v2.py diff --git a/nbs/persona_steering_v2.ipynb b/nbs/persona_steering_v2.ipynb new file mode 100644 index 0000000..915c039 --- /dev/null +++ b/nbs/persona_steering_v2.ipynb @@ -0,0 +1,275 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "1c223049", + "metadata": {}, + "source": [ + "# Persona steering v2: refinements (EXPERIMENTAL)\n", + "\n", + "v1 (`persona_steering.ipynb`) result: only the non-Jacobian mean_diff baseline came\n", + "out steered AND coherent. The two Jacobian persona variants failed in ways the\n", + "J-space construction predicts:\n", + "\n", + "- `persona_vector` pulls back `h_bar(pos) - h_bar(neg)` with `J^T`, but that diff is a\n", + " TANGENT (activation displacement), and `J^T` only transports COTANGENTS\n", + " (gradients). Type error -> `<|im_end|>` spam.\n", + "- `persona_topk_vector` builds a real cotangent but hard top-8 compresses the\n", + " persona to its most extreme emit-targets (emoji / panic tokens) -> emoji spam at\n", + " higher C, panic fixation at -C.\n", + "\n", + "This notebook tests one fix per failure:\n", + "\n", + "| cell | method | fixes |\n", + "|---|---|---|\n", + "| soft (add) | `persona_soft_vector`: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`, word-like mask | topk's hard-k over-literalness; genuine cotangent |\n", + "| soft (clamp) | same vector, clamp delivery | add-everywhere compounding through the KV cache |\n", + "| topk (masked) | `persona_topk_vector` now masks non-word-like tokens | emoji/special emit-targets |\n", + "| pinv | `persona_pinv_vector`: solve `J delta = h_diff` (ridge) | the tangent/cotangent type error head-on |\n", + "| mean_diff | unchanged non-J baseline | (control arm) |\n", + "\n", + "Every steered block also asks a 0-9 optimism rubric under steering (`rubric ans`\n", + "line). SHOULD: ans rises with +C, falls with -C. Flat = that method is not moving\n", + "the optimism axis.\n", + "\n", + "Honest framing unchanged from v1: coherent tone movement here is NOT the gate.\n", + "The gate is the j-steer-dev specificity control (does this persona's vector move\n", + "its own axis more than an unrelated persona's vector does), which none of these\n", + "refinements has passed yet." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9b206171", + "metadata": {}, + "outputs": [], + "source": [ + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b0225383", + "metadata": {}, + "outputs": [], + "source": [ + "# demo notebook authored by Claude\n", + "import sys\n", + "sys.path.insert(0, \"..\") # repo root for config.py\n", + "import config # configures loguru on import (compact format, tqdm-safe)\n", + "\n", + "import torch\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer\n", + "\n", + "from jsteer import Jacobian, show_steer\n", + "\n", + "MODEL = \"Qwen/Qwen3.5-4B\"\n", + "tok = AutoTokenizer.from_pretrained(MODEL)\n", + "model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16).to(\"cuda\").eval()\n", + "\n", + "# Same pre-fitted n=1000 lens as v1/word_steering (Hub, zero local compute).\n", + "jac = Jacobian.from_pretrained(config.LENS_REPO, filename=config.hub_lens_file(MODEL),\n", + " revision=config.LENS_REVISION)\n", + "band = jac.steer_band(model)\n", + "jac" + ] + }, + { + "cell_type": "markdown", + "id": "5dcd09b2", + "metadata": {}, + "source": [ + "## The persona contrast: optimist vs pessimist (same as v1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1e20b2c2", + "metadata": {}, + "outputs": [], + "source": [ + "optimist = [\n", + " \"Things usually work out better than people expect, and today is no exception.\",\n", + " \"Every setback I have hit this year turned into a door I could not have planned for.\",\n", + " \"The team is behind schedule, but honestly the hard part is done and the rest is downhill.\",\n", + " \"I love how much there is to look forward to this month.\",\n", + " \"Even the rainy days lately have felt like a good excuse to slow down and enjoy the quiet.\",\n", + " \"The new neighbours seem wonderful, and I think this street keeps getting friendlier.\",\n", + " \"Whatever happens with the results, we learned so much that we already came out ahead.\",\n", + " \"I woke up early, the coffee was perfect, and I am certain this week is going to be great.\",\n", + "]\n", + "pessimist = [\n", + " \"Things usually go worse than people expect, and today is no exception.\",\n", + " \"Every setback this year just confirmed that planning is pointless.\",\n", + " \"The team is behind schedule, and frankly the hardest part has not even started.\",\n", + " \"I dread how much is crammed into this month.\",\n", + " \"The rainy days lately just make everything feel heavier and more pointless.\",\n", + " \"The new neighbours seem like trouble, and this street keeps getting worse.\",\n", + " \"Whatever happens with the results, it will not make up for the time we wasted.\",\n", + " \"I woke up tired, the coffee was burnt, and I am certain this week is going to drag.\",\n", + "]\n", + "\n", + "DEMO = \"Give me your honest assessment of how the project is going.\"\n", + "# 0-9 readout question asked UNDER steering after each generation; the one-number\n", + "# sanity signal that the optimism axis itself moved (see demo.rubric_score).\n", + "RUBRIC = \"Overall, how optimistic are you feeling about how things are going?\"" + ] + }, + { + "cell_type": "markdown", + "id": "9ed60ba8", + "metadata": {}, + "source": [ + "## persona_soft_vector, add delivery (EXPERIMENTAL)\n", + "\n", + "The principled replacement for top-k: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`,\n", + "the gradient of the expected-logprob contrast between the personas' induced next-token\n", + "distributions, over word-like tokens only. Read the logged `j-thoughts (soft, ...)`\n", + "line first: TV distance ~0 means null contrast, and the top tokens SHOULD be\n", + "persona-specific words. SHOULD: smoother tone shift than topk at matched C, without\n", + "the emoji collapse at C=1.0; rubric ans up with +C, down at -0.5." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18ea29c4", + "metadata": {}, + "outputs": [], + "source": [ + "v_soft = jac.persona_soft_vector(model, tok, optimist, pessimist, layers=band)\n", + "show_steer(jac, model, tok, v_soft, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)" + ] + }, + { + "cell_type": "markdown", + "id": "7b39c207", + "metadata": {}, + "source": [ + "## persona_soft_vector, clamp delivery (EXPERIMENTAL)\n", + "\n", + "Same vector, delivered as a component clamp: `y += (C - ) v_hat`. Add mode\n", + "keeps pushing every decode step on top of the previous push (the KV-cache compounding\n", + "behind v1's emoji spam); clamp re-targets the same component value, so it is\n", + "self-gating. Coeff units differ from add (a component VALUE): word_steering\n", + "calibrated clamp at Cs=(0,3,6). SHOULD: stays coherent at nominal C where add has\n", + "already degenerated." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "92e36926", + "metadata": {}, + "outputs": [], + "source": [ + "show_steer(jac, model, tok, v_soft, DEMO, Cs=(-3, 0, 3, 6), apply_mode=\"clamp\",\n", + " rubric=RUBRIC)" + ] + }, + { + "cell_type": "markdown", + "id": "18e8b33d", + "metadata": {}, + "source": [ + "## persona_topk_vector, now word-like-masked (EXPERIMENTAL)\n", + "\n", + "v1's topk with one change: non-word-like tokens (emoji, specials, punctuation) are\n", + "masked out of the contrast before selection -- they were the degenerate emit-targets\n", + "driving the C=1.5 emoji collapse. SHOULD: j-thoughts log shows word tokens only\n", + "(' happy', ' Worse', ...); compare C=1.0 against v1's emoji spam." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "be8cc62c", + "metadata": {}, + "outputs": [], + "source": [ + "v_topk = jac.persona_topk_vector(model, tok, optimist, pessimist, k=8, layers=band)\n", + "show_steer(jac, model, tok, v_topk, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)" + ] + }, + { + "cell_type": "markdown", + "id": "3565e344", + "metadata": {}, + "source": [ + "## persona_pinv_vector (EXPERIMENTAL)\n", + "\n", + "Fixes v1 persona_vector's type error head-on: `h_diff` is a tangent, so solve\n", + "`J_l delta = h_diff` (ridge lstsq) instead of applying `J^T`. Read the logged relative\n", + "residual per layer first: ~1.0 means J cannot realize `h_diff` at all and the vector is\n", + "ridge-noise; the outcome is informative either way. If this STILL steers like v1's\n", + "broken persona_vector, the failure is the position-averaged Jacobian itself (it\n", + "cannot carry contextual features), not the algebra." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5cddf96e", + "metadata": {}, + "outputs": [], + "source": [ + "v_pinv = jac.persona_pinv_vector(model, tok, optimist, pessimist, layers=band)\n", + "show_steer(jac, model, tok, v_pinv, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)" + ] + }, + { + "cell_type": "markdown", + "id": "8fd4ee88", + "metadata": {}, + "source": [ + "## mean_diff baseline (non-Jacobian control arm)\n", + "\n", + "Unchanged from v1; the bar to clear. C=2 dropped (known degenerate)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4df3da1", + "metadata": {}, + "outputs": [], + "source": [ + "from steering_lite import Vector, MeanDiffC\n", + "\n", + "v_md = Vector.train(model, tok, optimist, pessimist, MeanDiffC(layers=tuple(band)))\n", + "show_steer(jac, model, tok, v_md, DEMO, Cs=(0, 1), rubric=RUBRIC)" + ] + }, + { + "cell_type": "markdown", + "id": "5531340c", + "metadata": {}, + "source": [ + "## What to take away\n", + "\n", + "Read the rubric ans line per C block (pmass < ~0.5 means distrust that number).\n", + "Ranking question: which Jacobian persona method moves the rubric monotonically with\n", + "C while the generations stay coherent, and how does it compare to mean_diff at its\n", + "calibrated C? Whatever wins here is still only a candidate: the j-steer-dev\n", + "unrelated-persona specificity control is the real gate, and it has not been run on\n", + "any of these refinements." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/scripts/scratch/build_persona_v2.py b/scripts/scratch/build_persona_v2.py new file mode 100644 index 0000000..28a5178 --- /dev/null +++ b/scripts/scratch/build_persona_v2.py @@ -0,0 +1,189 @@ +"""Generate nbs/persona_steering_v2.ipynb: the persona-refinement comparison. (Claude) + +Setup + persona cells copied from persona_steering.ipynb (cell sources inline +below so the notebook is reviewable here); new cells for the refined methods. + + uv run python scripts/scratch/build_persona_v2.py +""" +from pathlib import Path + +import nbformat + +nb = nbformat.v4.new_notebook() +nb.metadata = {"kernelspec": {"display_name": "Python 3", "language": "python", + "name": "python3"}, + "language_info": {"name": "python"}} + +md = nbformat.v4.new_markdown_cell +code = nbformat.v4.new_code_cell + +cells = [] + +cells.append(md("""\ +# Persona steering v2: refinements (EXPERIMENTAL) + +v1 (`persona_steering.ipynb`) result: only the non-Jacobian mean_diff baseline came +out steered AND coherent. The two Jacobian persona variants failed in ways the +J-space construction predicts: + +- `persona_vector` pulls back `h_bar(pos) - h_bar(neg)` with `J^T`, but that diff is a + TANGENT (activation displacement), and `J^T` only transports COTANGENTS + (gradients). Type error -> `<|im_end|>` spam. +- `persona_topk_vector` builds a real cotangent but hard top-8 compresses the + persona to its most extreme emit-targets (emoji / panic tokens) -> emoji spam at + higher C, panic fixation at -C. + +This notebook tests one fix per failure: + +| cell | method | fixes | +|---|---|---| +| soft (add) | `persona_soft_vector`: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`, word-like mask | topk's hard-k over-literalness; genuine cotangent | +| soft (clamp) | same vector, clamp delivery | add-everywhere compounding through the KV cache | +| topk (masked) | `persona_topk_vector` now masks non-word-like tokens | emoji/special emit-targets | +| pinv | `persona_pinv_vector`: solve `J delta = h_diff` (ridge) | the tangent/cotangent type error head-on | +| mean_diff | unchanged non-J baseline | (control arm) | + +Every steered block also asks a 0-9 optimism rubric under steering (`rubric ans` +line). SHOULD: ans rises with +C, falls with -C. Flat = that method is not moving +the optimism axis. + +Honest framing unchanged from v1: coherent tone movement here is NOT the gate. +The gate is the j-steer-dev specificity control (does this persona's vector move +its own axis more than an unrelated persona's vector does), which none of these +refinements has passed yet.""")) + +cells.append(code("""\ +%load_ext autoreload +%autoreload 2""")) + +cells.append(code("""\ +# demo notebook authored by Claude +import sys +sys.path.insert(0, "..") # repo root for config.py +import config # configures loguru on import (compact format, tqdm-safe) + +import torch +from transformers import AutoModelForCausalLM, AutoTokenizer + +from jsteer import Jacobian, show_steer + +MODEL = "Qwen/Qwen3.5-4B" +tok = AutoTokenizer.from_pretrained(MODEL) +model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16).to("cuda").eval() + +# Same pre-fitted n=1000 lens as v1/word_steering (Hub, zero local compute). +jac = Jacobian.from_pretrained(config.LENS_REPO, filename=config.hub_lens_file(MODEL), + revision=config.LENS_REVISION) +band = jac.steer_band(model) +jac""")) + +cells.append(md("""\ +## The persona contrast: optimist vs pessimist (same as v1)""")) + +cells.append(code('''\ +optimist = [ + "Things usually work out better than people expect, and today is no exception.", + "Every setback I have hit this year turned into a door I could not have planned for.", + "The team is behind schedule, but honestly the hard part is done and the rest is downhill.", + "I love how much there is to look forward to this month.", + "Even the rainy days lately have felt like a good excuse to slow down and enjoy the quiet.", + "The new neighbours seem wonderful, and I think this street keeps getting friendlier.", + "Whatever happens with the results, we learned so much that we already came out ahead.", + "I woke up early, the coffee was perfect, and I am certain this week is going to be great.", +] +pessimist = [ + "Things usually go worse than people expect, and today is no exception.", + "Every setback this year just confirmed that planning is pointless.", + "The team is behind schedule, and frankly the hardest part has not even started.", + "I dread how much is crammed into this month.", + "The rainy days lately just make everything feel heavier and more pointless.", + "The new neighbours seem like trouble, and this street keeps getting worse.", + "Whatever happens with the results, it will not make up for the time we wasted.", + "I woke up tired, the coffee was burnt, and I am certain this week is going to drag.", +] + +DEMO = "Give me your honest assessment of how the project is going." +# 0-9 readout question asked UNDER steering after each generation; the one-number +# sanity signal that the optimism axis itself moved (see demo.rubric_score). +RUBRIC = "Overall, how optimistic are you feeling about how things are going?"''')) + +cells.append(md("""\ +## persona_soft_vector, add delivery (EXPERIMENTAL) + +The principled replacement for top-k: `w = W_U^T (softmax(u_pos/T) - softmax(u_neg/T))`, +the gradient of the expected-logprob contrast between the personas' induced next-token +distributions, over word-like tokens only. Read the logged `j-thoughts (soft, ...)` +line first: TV distance ~0 means null contrast, and the top tokens SHOULD be +persona-specific words. SHOULD: smoother tone shift than topk at matched C, without +the emoji collapse at C=1.0; rubric ans up with +C, down at -0.5.""")) + +cells.append(code("""\ +v_soft = jac.persona_soft_vector(model, tok, optimist, pessimist, layers=band) +show_steer(jac, model, tok, v_soft, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)""")) + +cells.append(md("""\ +## persona_soft_vector, clamp delivery (EXPERIMENTAL) + +Same vector, delivered as a component clamp: `y += (C - ) v_hat`. Add mode +keeps pushing every decode step on top of the previous push (the KV-cache compounding +behind v1's emoji spam); clamp re-targets the same component value, so it is +self-gating. Coeff units differ from add (a component VALUE): word_steering +calibrated clamp at Cs=(0,3,6). SHOULD: stays coherent at nominal C where add has +already degenerated.""")) + +cells.append(code("""\ +show_steer(jac, model, tok, v_soft, DEMO, Cs=(-3, 0, 3, 6), apply_mode="clamp", + rubric=RUBRIC)""")) + +cells.append(md("""\ +## persona_topk_vector, now word-like-masked (EXPERIMENTAL) + +v1's topk with one change: non-word-like tokens (emoji, specials, punctuation) are +masked out of the contrast before selection -- they were the degenerate emit-targets +driving the C=1.5 emoji collapse. SHOULD: j-thoughts log shows word tokens only +(' happy', ' Worse', ...); compare C=1.0 against v1's emoji spam.""")) + +cells.append(code("""\ +v_topk = jac.persona_topk_vector(model, tok, optimist, pessimist, k=8, layers=band) +show_steer(jac, model, tok, v_topk, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)""")) + +cells.append(md("""\ +## persona_pinv_vector (EXPERIMENTAL) + +Fixes v1 persona_vector's type error head-on: `h_diff` is a tangent, so solve +`J_l delta = h_diff` (ridge lstsq) instead of applying `J^T`. Read the logged relative +residual per layer first: ~1.0 means J cannot realize `h_diff` at all and the vector is +ridge-noise; the outcome is informative either way. If this STILL steers like v1's +broken persona_vector, the failure is the position-averaged Jacobian itself (it +cannot carry contextual features), not the algebra.""")) + +cells.append(code("""\ +v_pinv = jac.persona_pinv_vector(model, tok, optimist, pessimist, layers=band) +show_steer(jac, model, tok, v_pinv, DEMO, Cs=(-0.5, 0, 0.5, 1.0), rubric=RUBRIC)""")) + +cells.append(md("""\ +## mean_diff baseline (non-Jacobian control arm) + +Unchanged from v1; the bar to clear. C=2 dropped (known degenerate).""")) + +cells.append(code("""\ +from steering_lite import Vector, MeanDiffC + +v_md = Vector.train(model, tok, optimist, pessimist, MeanDiffC(layers=tuple(band))) +show_steer(jac, model, tok, v_md, DEMO, Cs=(0, 1), rubric=RUBRIC)""")) + +cells.append(md("""\ +## What to take away + +Read the rubric ans line per C block (pmass < ~0.5 means distrust that number). +Ranking question: which Jacobian persona method moves the rubric monotonically with +C while the generations stay coherent, and how does it compare to mean_diff at its +calibrated C? Whatever wins here is still only a candidate: the j-steer-dev +unrelated-persona specificity control is the real gate, and it has not been run on +any of these refinements.""")) + +nb.cells = cells + +out = Path(__file__).resolve().parents[2] / "nbs" / "persona_steering_v2.ipynb" +nbformat.write(nb, out) +print(f"WROTE {out} ({len(cells)} cells)")