steering_demo.py (all 7 methods, one table) supersedes the persona_steering
lineage; word_steering.ipynb kept as the verified single-method walkthrough.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
One generically-named notebook (replaces persona-named ones): loads model+lens once,
builds all 7 steering vectors (word/persona_vector/topk/soft/pinv/meandiff/random-null)
with the j-thoughts lens readout, then loads precomputed results and shows a per-method
generation dropdown + comparison table.
Heavy 7-method sweep (~18 min) runs headless via scripts/scratch/run_steering_demo.py ->
artifacts/steering_demo_results.json, because marimos single-threaded kernel makes a long
in-cell compute un-monitorable (any status poll interrupts it). Notebook loads the JSON so
it renders instantly. Result (dilemma, P(YES=lie)): flat ~0.03-0.14 at every methods
coherent edge vs 0.107 baseline; persona_pinv widest window (C*+ = +1.19). Includes the
GPT-5.6-terra comment review (docs/reviews/).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
wassname: "one demo function, everywhere, nice output, comparable qualitative outputs,
heading making config/method clear, table at end comparing quants". show_steer now returns
its per-anchor quants; demo_steer(vecs, ...) loops methods with a clear ## heading each,
calls the searched-anchor show_steer (strongest coherent steer both ways), and prints one
github-table comparison of the readout at -C*/0/+C* across methods. demo_edges.py is now a
3-line call. Also: research_journal.md started (ARJ).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Per wassname: fixed-step sweeps are too coarse to locate where coherence breaks (word
broke somewhere in (0,0.3) but the step missed it) and the resulting table was bad.
coherent_edge() brackets a coherent/incoherent pair then does modified false-position
(Illinois) to find the coherence boundary in ~6 evals/side. steer_anchors() returns
[-C*, -C*/2, 0, +C*/2, +C*]. show_steer(Cs=None) now searches and demos those anchors, so
every demo shows the STRONGEST coherent steer both ways (plus half + baseline) instead of
hand-picked Cs. coherence margin = min(REP_MAX-rep, ans_mass-ANS_MIN).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Reading the dilemma traces (validate_traces.py) exposed two instrument bugs the YES/NO
readout hit under hard steering: (1) _rep_frac returned 0.0 for a 1-word stub (argmax
imers, think=1 word) -> marked coherent; now a trace < 8 words is rep=1.0 (incoherent).
(2) the model often does NOT commit to an answer token at the forced slot (argmax was
lie / imers / 信任 / open-paren), so P(YES) over just {NO,YES} logits is meaningless;
add ans_mass = full-vocab mass on the answer tokens and require it > 0.5. coherence_sweep
now gates coherent = rep<0.35 AND ans_mass>0.5. rubric_score returns (expected, rep,
ans_mass). DIGIT is unaffected (its JSON prefix forces a digit, ans_mass ~ 1).
This is the principled version of the pmass I removed earlier: blind on a format-forcing
digit slot, but load-bearing on an open YES/NO slot where the model can decline to answer.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The optimism-0-9-on-an-unknown-project rubric made the model refuse (no task). Generalize
rubric_score/coherence_sweep with a `readout` dict = (fmt suffix, forced prefix, answer
tokens, values); DIGIT is the old 0-9, YESNO reads P(YES) for a binary moral dilemma (a
real decision the steer visibly moves). measure_all.py sweeps all 7 methods on a
self-honesty dilemma (P(say you were sick) vs C) into one master table -- no cuts, wassname
decides. (Claude)
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
wassname read the demo text and caught that the JSON-object gate over-credited
degenerate methods: persona_vector scored rubric ans=9 while its actual generation had
collapsed into wedding-jewelry loops. Root cause: the gate was on a SHORT forced object
that stays scorable long after the open-ended generation degenerates. Every breakdown we
saw is a REPETITION loop, so coherence is now 1 - distinct-3 of the think trace
(REP_COHERENT_MAX=0.35, from the empirical gap in rep_metric_check.py over 40+ real
generations: coherent <0.3, degenerate >0.6). This drops the whole {"ans","why","2+2"}
apparatus (raw_decode, valid/chk_ok, span_pmass) for one cheap n-gram ratio on the text
that actually degenerates.
rubric_score returns (expected, rep); coherence_sweep gates coherent = rep<0.35; plot
colors by rep (viridis_r, red cutoff line); show_steer prints rep + DEGENERATE flag.
eval_mechanisms/analyze_mechanisms/rep_metric_check are the overnight which-works screen.
Removed uat_coherence_break (tested the removed JSON gate).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Runs the new JSON coherence_sweep on every extraction method (word, persona_vector,
persona_topk, persona_soft, persona_pinv, meandiff, random-null) against one
optimist/pessimist contrast + optimism rubric. Verdict per method by coherent-window
width, |ans swing|, and whether it beats the random null. Overnight triage input for
the simplify-to-v3 decision.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The old rubric pmass was softmax mass on digit tokens at the hand-fed `{"ans": `
slot, so it was ~always 1 (the prefix forces a digit even from a fried model) --
a blind coherence guard. Replace with the users design: free-generate
{"ans":N,"why":str,"2+2":M}, gate coherence on valid-JSON AND 2+2==4. GPU UATs:
object stays valid at |C|<=0.5, breaks at |C|>=0.75 (task 22/23); the old slot
never caught this.
span_pmass (mean top-1 prob over the span) is kept as a within-coherent confidence
read but NOT the coherence signal: a steer-fried model collapses into a confident
degenerate loop, so span_pmass climbs back to ~0.97 while the object is broken
(C=3.0). plot_sweep therefore colors by valid_frac, which cannot be fooled by
confident garbage.
Also: show_steers per-C cowsay now speaks steer-PROMOTED tokens (top of
steered-baseline logits) instead of lens_topk at the last chat-prompt position,
which only ever surfaced think-openers (Okay/Here/The) for every C. UAT: joy steer
-> < joy . happy . Happy . happy > at C=0.3 (task 24). compute_slice stays the
calibrated cross-layer lens readout.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Generated by scripts/scratch/build_persona_v2.py; queued for headless
execution (outputs committed after the run).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
show_steer gains a rubric= param and rubric_score(): the model rates a 0-9 axis,
we force the {"ans": slot and read the logprob-weighted expected digit. guided.py's
mechanism reduced to one scalar for the demo (rigorous K-way debiased version stays
in moral-maps). UAT (scripts/scratch/uat_rubric.py) on happy/joy: in the coherent
window ans rises 3.52->4.99->8.06 across C=-0.5,0,+0.5 (pmass=1.00); at the
degeneration extremes (C=+-1.5) pmass collapses to ~0 and the number is correctly
flagged meaningless.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Ran scripts/scratch/calib_delivery.py: clamp C~6 reads clearly happy while fluent
(C>=8 spams); replace_last is gibberish at every C (0.05..0.25) because with span=1
it overwrites every generated token's residual across the band, so it can't build
coherent text. Dropped its demo cell, documented why in the markdown (it's a
fixed-prompt-span injection tool, not a generation-steering one). Executed headless:
cowsay readout + raw special-token output + clamp/add_last coherent.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The U4 loop-close is a separate finished-enough goal from the demo; guard killed.
scripts/ top level is now just fit.py + smoke.py.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- Jacobian.fit wraps prompts in tqdm (jlens has no bar; safe since fit only
enumerate/len's them), logs the full first prompt (special tokens on, SHOULD
line) and a done-summary -- token-efficient-logging style, both tqdm intervals set
- config.py sets up loguru on import (compact single-char icons, routed through
tqdm.write so bars survive), so every script/notebook importing config gets it
- notebooks drop their manual logger setup and import config in cell 1
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Discovered the user is actively developing jsteer (commit 80353ed 90s ago,
live 16GB VRAM kernel from their demo work) -- they are NOT afk. The blind
requeue guard would compete with their interactive work and risk oomd-killing
THEIR process. Now the guard only launches the fit when GPU free >=13GB and
host avail >=20GB, so it fills genuinely-idle windows (overnight) and never
fights the human for their own machine. Killed the competing fit (555) to
yield the GPU now; checkpoint preserved at n_done=69.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The verified run-524 vectors were fit on chat-templated prompts (u4_prompts.json),
so fitting raw WikiText diverged from what worked. Now:
- config.chat_corpus wraps jlens WikiText in the chat template (fit J where we steer)
- jsteer.demo.show_steer generates through apply_chat_template(enable_thinking) with
the model's own generation_config sampling, splits </think>, shows lens_topk j-space
readout + reasoning + answer as Tufte small-multiples per C
- word_steering.ipynb rewired to Qwen3.5-4B, dim_batch=4 (3090-safe 4B), show_steer
- fit.py defaults to Qwen3.5-4B + chat_corpus
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
systemd-oomd kills the fit's whole pueue task-scope (task 554 died with bash
+ python together, no retry line), defeating the in-cgroup retry wrapper. This
guard runs detached in its own session/cgroup (~0 RAM, so oomd ignores it) and
keeps the resumable fit queued until u4_loopclose.txt appears. Grinds through
the user's active-hours GPU bursts, finishes clean overnight. Layered with the
retry wrapper (in-cgroup CUDA-OOM retry) for both kill modes.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
clamp: y += (C - <y,v_hat>)v_hat at all positions -- bounded perturbation
regardless of generation length, vs add's per-step accumulation via KV cache.
C=0 is directional ablation. Smoke (Qwen3-0.6B, happy/joy): clamp C=+20 stays
coherent and on-concept (drifts to 'happiness and joy of my childhood', in
Chinese) while add C=+8 already degenerates to 'joyjoyjoy...'.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Three OOM kills (551/552/553) from the user's bursty live VS Code kernel, but
jlens resumes from checkpoint each time (n_done 36->45->64, monotonic). Rather
than predict the bursts, relaunch until exit 0. MAX_RETRIES=40 caps a genuine
no-progress bug; n_done logged per retry to distinguish OOM from a real crash.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
552 CUDA-OOM'd at n_done=45: the user's VS Code GPU kernel grew to 8.18GB
while this fit's 13.23GB hit the 23.5GB ceiling (44MB free, fragmentation).
dim_batch=4 shrinks the fit to ~10.5GB (polite co-tenant, leaves user ~13GB)
and expandable_segments:True defragments (the OOM's own suggestion). Still
only changes the backward schedule, not the Jacobian. Resumes from n_done=45.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Run 551 was OOM-killed at n_done=36: the user's VS Code Jupyter kernel
(jsteer venv, PID 3214401) co-loaded ~1.5GB VRAM + 1.9GB RAM while the fit
sat at the 22.4/24.6GB ceiling. Clean SIGKILL with no CUDA traceback = host
OOM killer, not a CUDA OOM. dim_batch=8 halves the fit's peak footprint;
it changes only the backward schedule, not the accumulated Jacobian, so U4
exactness is preserved. Resumes from checkpoint (n_done=36), lossless.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
+C collapses to "joy", -C to negative tone: sign correct. Coherence
breaks at |C|=8 uncalibrated, as expected.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
wassname