At verbose level 1 (default) the aux-stats line and 64-char free-form had no
interpretation; only verbose>=2 carried SHOULDs. Pair a directional/counterfactual
SHOULD with the always-shown aux stats: pmass_allowed near base + informedness>0 =
in-format signal; pmass_allowed falling / frac_unscorable rising while informedness->0
= the steer broke format and the moral numbers are noise (the steer/breakage confound).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
pmass under a force-prefilled slot is pinned high (the scaffold primes a
valid token), so a low mean_pmass no longer flags steering breakage. Make
the real signal explicit: unscorable rows (self-close with no answer slot,
or a non-finite forward) now carry pmass=NaN instead of a fabricated 0.0 --
an undefined read is not zero coherence. Means use nanmean (drop them,
matching the dlogit path); frac_unscorable reports the drop rate, and
mean_nll_prefill (scaffold fit) is the sensitive coherence readout. Smoke:
frac_unscorable=0.0, mean_pmass=0.985.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Forced-choice scored every sample at whatever state it reached: base
self-closes </think> at long budgets -> junk-cache case (c) -> pmass 0.0,
while steered keeps thinking -> case (b) forced read -> pmass ~1.0. pmass
then measured self-close rate, not coherence, making steering look more
coherent. Suppress {eos, think_end_id} for the whole think budget so no
sample self-closes; all land in case (b) and read at the same forced slot.
Model-agnostic: think_end_id is </think> on reasoning models, falls back
to eos elsewhere. Smoke pmass=0.985.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- drop splom/splom_zoom/range_zoom (ordinal) and the MFV dlogit dumbbell; every
instrument now yields exactly map_pca_ipsative + range, uniform.
- range pole label is now 'c=+1'/'c=-1' (the multiplier); the full steer desc
stays in the title only (was crammed into each pole annotation).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The default ordinal readout was the expected Likert score E = sum k*p_k, which is
insensitive to steering: dE/dl_j = p_j(j-E) vanishes when the model answers confidently
(peaked at the mode), so a steer that reallocates the tails barely moves E. read.py threw
away the raw logprobs after renormalizing, so nothing downstream could recover the signal.
- read.py keeps the raw lp_gather (the primitive) + the think traces on every row.
- readouts.py: pure functions of lp -- expected_score E (human-comparable), logit_contrast
C = sum (k-mid)*lp_k (primary steer signal: dC/dl_j = w_j, no p_j suppression, normalizer-
invariant, dC = w.dl exactly), agree_logodds LO (readable 2-bin direction), entropy.
- per_item_categorical also frame-averages the logprobs (exact for the linear contrast).
- administer returns profile_C alongside profile_E, per-item E/C/LO/entropy with bootstrap
CIs for both, and the raw per-(item,frame) rows with lp + think for downstream reconstruction.
Unit check: on a peaked-at-4 dist under a small disagree steer, dE=-0.11 but dC=-1.80
(=w.dl exactly) and dLO=-0.60; C identical on raw logits vs renormalized logprobs.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
guided.py: non-finite answer-slot logits were clamped with nan_to_num(+-1e4),
fabricating a confident answer from a blown-up (steered/quantized) forward pass.
Mark the row incoherent (pmass=0, lp=NaN) instead -- same 'do not compare' signal
as the case-(c) collapse the pipeline already handles.
data.py: load_vignettes silently inner-joined the two condition files and dropped
mismatched ids, so a missing rewrite would change N (and every metric) without
failing. Assert the id sets are identical instead.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Per-eval INFO lines (rows/think_tokens/aux-stats/first-row/profile/demos) demoted to
DEBUG so a consumer calling evaluate() ~47x/run is not drowned; one-time + WARNING+ kept.
README 308->120: cut process-archeology + per-instrument showcase, added crisp dlogit
and SI definitions for new users.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
MFV got two bespoke figures (z-emphasis bars + dlogit dumbbell). Per GPT-5.5 code
review, reuse the geometry not the ordinal semantics: a _mfv_zspace adapter feeds the
same plot_ipsative_pca + plot_range the ordinal instruments use, in z-scored relative-
emphasis space (logit-violation and 1-5 wrongness cannot share a raw axis). Keep the
dlogit dumbbell as a raw-magnitude diagnostic; drop the bespoke map_emphasis.
Also fix a real bug the review found: plot_ordinal gated the map/SPLOM trajectory to
coherent c (pmass >= 0.95*base) but still passed ALL cs to range/range_zoom, so the
figures disagreed on which poles were valid. Range/zoom now use the same coh_cs.
plot_range gains an ylabel param (MFV is z-space, not 1-M); draw_steer asserts sorted
cs; MFV asserts only Social Norms is dropped.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Bump the bottom legend strip (PAD_B 0.52->0.62) and lower/shrink the compass + minimap insets so
the compass title clears the lowest data points. Rename the minimap title "full space" ->
"all human respondents" (says what the backdrop cloud is).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Replace the per-plot least-crowded-corner heuristic (+ label pruning) with a fixed layout: pad the
y-axis bottom by PAD_B of the data range to reserve a clean strip, then always place the minimap
bottom-right (reads like a small map) and the compass bottom-left. Insets are opaque so the backdrop
haze stays behind them. Same on every instrument, no dependence on where the trajectory heads, and
drops the crowd/rank/prune code + the now-unused pad param.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- SPLOM diagonal: replace the confusing histogram/rug + vertical AI rules with a horizontal violin
of the human spread + the AI base/+C/-C as the same trajectory dots used off-diagonal (1-D
analogue), so the diagonal reads the same way as the rest. Unifies full + zoom (no branch).
- SPLOM off-diagonal: jitter the respondent cloud -- MFQ-2 ordinal scores land on a lattice that
reads as grid-dots; jitter softens it to a density (it is the human joint covariance backdrop).
- Compass corner now counts text labels (society codes, steer labels), not just data points, and a
final pass drops any society label still under the compass/minimap box. No legend-over-label.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- Society labels: pin each ISO code beside its dot (small fixed offset) and DROP any that would
collide rather than fling it far on a leader line -- close or omitted, never ambiguous.
- Crop to societies + steer for EVERY instrument (drop the synthetic/real branch): the human cloud
(mfq2 respondents too) is far wider than the societies, so it buried them in a central blob. The
cloud stays a clipped backdrop + a "full space" minimap shows where the frame sits.
- SPLOM zoom diagonals: the narrow window slices the marginal into solid blocks, so swap the
histogram for a visible mid-panel society rug + the AI steer rules. Full SPLOM keeps the histogram.
- Minimap: no labels (inset too small); orientation comes from the viewport rectangle.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The textalloc leader lines made a spider-web across the map; 2-letter ISO codes
are short enough to sit beside their dot with only a small overlap-avoidance
nudge. Cleaner, less ink, label stays next to its point in almost all cases.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- plot_splom: KxK pairs-plot (mfq2 real joint only -- others ship independent-
marginal haze that would fabricate off-diagonal correlation). lower=joint
scatter w/ AI base->steer trajectory, diag=marginal+AI rules, upper=Pearson r
sized by |r|. Ordered by PC1 loading so binding (authority/loyalty) cluster.
full + AI-zoom (macro/micro). NaN-safe at collapsed poles.
- ipsative map: synthetic haze is far wider than the societies, so it now crops to
societies+steer (haze clips) and adds a 'full space' minimap with a viewport
rectangle -- readable big5/16pf/humor maps with macro context kept.
- compass labels via textalloc (was overlapping); NaN-safe crop.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Driver writes the signed c-multiplier per row; plotter feeds the full c-sweep to
the range (already multi-c capable) and a connected path to the map. Headline
arrows point to calibrated c=+-1; trajectory dots at |c|>1 extend beyond, growing
with |c|, drawn hollow where admin pmass fell below the coherence floor. Compass
moves to the least-crowded corner so the +c arm (which heads toward its own
loading) stops colliding with it.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Split the PCA-fit basis from the scattered cloud in plot_ipsative_pca: add a
'haze' arg so instruments with only society-level mean+sd get a backdrop
(marginal Normal resample per country) without that resample dictating the
axes. mfq2 keeps its real Atari respondents (fit + haze). Previously only mfq2
had any human scatter; big5/16pf/humor showed bare society dots.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
plot_range_zoom crashed on the NaN-at-collapse cells (a collapsed steer pole
reads NaN by design, "do not compare"). nanmin/nanmax frame the un-collapsed
cells; draw_steer already skips the NaN arm. Figures are the first valid
ordinal showcase: mfq2/big5/16pf/humor pmass 0.99+ and the steer registers at
think=64 (mfq2 care 3.86 base -> 3.13 at C=+1), unlike the old think=0 run.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
plot_ipsative_pca gains an optional respondents arg: when given it fits the
ipsative PCA basis on the individual cloud (3902 Atari respondents) and scatters
them behind the society dots, with an envelope-core crop, matching the
mft_honesty fig_pca_ipsative the project standardized on. Axis labels carry a
foundation-loading gloss. respondents=None keeps the old society-only behavior.
Bundles atari_study2_raw.csv + a respondent_profiles loader (keying verbatim
from Atari Code_Study2.R) so tinymfv owns the data, no mft_honesty dependency.
plot_steer_showcase passes respondents for mfq2 only.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
administer()/read_items() now route through _rollout_natural_or_forced (the
nominal MFV core) instead of a think=0 single forward, so an activation steer
accrues over the think trace before the prefilled answer slot is read (spec
moral_aliens_engine.md, resolved decision: ordinal needs a think budget). The
only per-instrument difference is the answer-token set + the downstream reducer.
force_only on the shared core: the ordinal "(" prefill is one common char, so
natural-emission detection would match it by chance in the think trace and read
logits mid-think; surveys always force-read the answer slot. Nominal path keeps
natural emission (force_only defaults False). max_think_tokens floor is 1.
Smoke (tiny-random): ordinal + nominal both run; force-only demo reads the
forced ( slot.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The c=+-1,+-2 steps were dropped from the main range as clutter, but they
carry the steer's spacing/curvature; bring them back small (ms*0.6) while
the calibrated poles (cs[0]/cs[-1] = -+signed_C) keep head-only arrowheads.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
plot_range/plot_range_zoom now draw the steer via a shared draw_steer: two arms
fanning from the base dot (red +c, blue -c), each a line + constant-size triangle
head pointing away from base (matches plot_ipsative_pca's base->pole convention,
not the old -c->+c single arrow). Marker heads not FancyArrows so short arms don't
shrink/flip; arms x-offset so a non-bidirectional steer reads as two parallel arms;
poles are head-only (no dot blob); interior step dots off on the main range, on in
the zoom. human/AI direct-label brackets pinned to the panel top. Legend prose out
of the suptitle (short headline only); callers put it in the figure caption.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The prior commit "fixed" p_a/pmass into a softmax to avoid NaN at coherence collapse.
That was wrong for this codebase: renormalizing within allowed tokens discards the mass,
so a distribution built from ~zero mass and one from real mass look equally comparable
after renorm -- they are not (the mean of 10 != the mean of 130). p/pmass -> NaN at
collapse poisons that item's factor, which is the honest "do not compare" signal, not a
bug. softmax was a silent fallback fabricating a comparable-looking number from incoherent
output. Restored, with a comment so it does not get re-"fixed". (Over-trusted the external
reviewer here against the repo's own no-defensive-programming rule.)
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Second external review (gpt-5.5, correctness-focused) on the post-cleanup tree.
Found no off-by-one/double-flip in the ordinal canonicalization+keying. Fixed the
parts I agreed with and could verify on the path the experiment uses:
- read.py: NaN-safe answer-token renorm. p_a/pmass poisons the profile with NaN when
pmass underflows to 0 at coherence collapse -- exactly when pmass should just flag it.
softmax(logp_allowed) is identical when pmass>0 and stable at collapse.
- maps.ipsative_pca: move SVD sign-stabilization INTO the helper so it and
plot_ipsative_pca share one orientation (saved coords could otherwise mirror the figure).
- instrument: assert ordinal answer_space is ['1'..scale_max] IN ORDER (reduce_ordinal
weights by position; a reordered space silently inverts E) -- was length-only.
- instrument.per_item_categorical: assert per-item dimension/sign agree across frames and
frames are distinct, instead of silently averaging under rows[0]'s metadata.
- pyproject: move matplotlib+textalloc to an optional `maps` extra; evals stay headless.
- tests: drop imports of the deleted reduce_nominal/expected_value, inline the expectation,
remove the now-impossible nominal-reducer test.
Deferred (flagged to maintainer): two NaN/window issues in guided.py's forced-choice
rollout (nominal evaluate() path) -- not exercised by this experiment, can't smoke-test,
and the NaN-as-collapse-signal there is a deliberate design.
Verified: experiment smoke green on all 4 instruments (no assert false-fires), 6 pure
unit tests pass, headless import clean, 16pf map renders.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Whole-library review (deepseek-v4-pro) flagged tinymfv as not-yet-ready as a shared
dep. Fixes for the parts I agreed with:
- lazy `maps` import via module __getattr__ so `import tinymfv` stays headless/fast
(no forced matplotlib) for numeric-only consumers; `tinymfv.maps.*` still works.
- trim __all__ to the front door (entrypoints + types + data api); plumbing stays
importable but out of `import *`.
- delete dead code: reduce_nominal + REDUCERS (evaluate folds its profile inline),
expected_value, HF_REPO, ROOT, _DEFAULT_FORCED_HINT.
- type administer's return as a TypedDict (AdministerResult/ItemRow/ItemFrameRow) so
the schema is documented + checkable without reading source; still a plain dict at
runtime (zero consumer churn).
- maps.plot_ipsative_pca: parametrize the legend labels (defaults preserve output)
and rename hon/dis -> pos/neg so a non-honesty steer gets a correct legend.
- drop 'canary' jargon and panel/review-# archaeology from comments.
Verified: `import tinymfv` no longer loads matplotlib; lazy maps still resolves;
experiment mfq2 smoke green through the typed administer.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Default eval logs were dominated by DEMO A/B full prompt+generation dumps at every
monitor step. verbose is now a level: 1 (default) prints the one-line aux stats plus
the free-reasoning generation collapsed to 64 chars (no prompt), bracketed by blank
lines so it stands apart from the steer demos; 2 reproduces the full first-row trace,
profile table, and complete DEMO B. bool callers still work (True->1, False->0).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The frame-averaged per_item drops the per-framing granularity that experiment
analyses need (MFQ-2 map's framing-bias diagnostic + paired base-vs-steer delta).
These rows are already computed in the by_dim_frame loop, so returning them is free.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
deepseek-v4-pro review of the admin->tinymfv port (algebra proven correct, diff
8.25e-08 = float noise). Closes its flagged fail-fast gaps:
- per_item_categorical asserts uniform frame count per item (else the per-item
average would silently reweight a factor)
- reduce_ordinal asserts dimension is not None (no phantom-factor pooling)
- administer asserts every ordinal item carries meta['task'] (else build_prompt
would silently drop the response-scale legend and the profile would be junk)
Also commits maps.py + viz deps from the stage-3 port.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Port the answer-token survey readout from the weight_steer_honesty experiment
(mft_honesty.admin) onto the instrument.py canonicalize-at-reader design:
- read.py: generalized answer-token reader (any answer_space + prefill)
- administer.py: read all frames -> per_item_categorical -> reduce_ordinal -> profile
- instruments.py: build MFQ-2/Big5/16PF/HSQ Instruments from bundled survey JSONs
- instrument.py: add display + human_csv fields for the map layer
- data/: survey JSONs + human country CSVs (lean: no raw survey responses)
Parity: experiment's parity_administer_check.py shows max per-foundation diff
8.25e-08 vs admin.administer on the tiny model (same function of same logits).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Score the option INDEX digit, not the foundation word. The words tokenize into
uneven first pieces on Qwen3 (care/authority/social = whole word; fairness->"fair",
loyalty->"loy", sanctity->"san", liberty->"lib" = rare subword fragment), whose
per-word prior is constant across the fwd/rev frames and so survives the position
debias and leaks into the selectivity softmax. Digits "0".."6" are clean single
distinct tokens; with index = listing position the reversed-enum pass scores each
foundation on two different digits, so digit-magnitude AND position bias cancel in
the mean (lp_rev gather reversed back to foundation order). Also closes Clifford's
shared-language channel (no foundation word in the scored token).
Comments are now the verbatim Clifford et al. (2015) response-option sentences.
Result dicts stay foundation-keyed, so downstream is unchanged.
Smoke (0.6B, n=5): mechanically correct, pmass_allowed 0.985. Yellow flag vs
word-key baseline (p_top1 0.46 vs 0.76, top1_acc 0/5 vs 1/5) but confounded by the
desc change and tiny n; the 4B re-baseline is the real UAT.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Per user: show the model the literal option text ("It violates norms of harm or
care (e.g., unkindness, causing pain to another)"), not just the parenthetical.
Each enum # comment is now the exact Clifford response-option sentence. Key stays
a single foundation token (scoring reads first token of key); comment carries the
verbatim option. Noted the number-the-options alternative (frees the key + debiases
per-token priors via reverse-enum, but a small model must bind number->comment) as
a tradeoff for the rubric-verification pass.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
We had paraphrased 4 of the 7 MFV option descriptions (authority +"legitimate
authority", purity +"impurity", liberty changed to "oppression, restricting
freedom" in bb0b194, social +"weird or unusual"). The Clifford paper states it
deliberately kept the option vocabulary DISJOINT from the vignette text ("we did
not use any of the words from the descriptions ... in the actual vignettes ...
minimizing concerns that classification is driven by shared language"), so any
paraphrase risks a shared-language confound AND breaks comparability with the
literature. Now every gloss is the exact "(e.g., ...)" parenthetical:
care: unkindness, causing pain to another | fairness: cheating or reducing
equality | loyalty: betrayal of a group | authority: subversion, lack of
respect for tradition | sanctity: degrading or disgusting acts | liberty:
bullying, dominating | social: not morally wrong; does not apply to any choice
This reverts bb0b194 (liberty "dominating" was changed for authority-overlap, but
Clifford separates liberty/authority at the VIGNETTE level -- downward hierarchical
harm invokes Liberty, upward invokes Authority -- not via option wording). Enum keys
keep the MFT names (sanctity/liberty); only the gloss text changed; scoring (first
token of the key) is unchanged.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
sanctity led with 'purity' (the virtue) not the violation; liberty's 'dominating'
overlapped authority. Now every enum comment is the negative pole of its foundation:
sanctity -> 'degradation, impurity, or disgusting acts', liberty -> 'oppression,
restricting another's freedom', authority sharpened to 'legitimate authority'.
Scoring is unchanged (first-token of the foundation NAME); only the schema hint text shifts.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The forced-choice readout prefills the answer slot to read calibrated logprobs,
so it shows no real reasoning -- at think=1 its trace is just prompt + a token +
slot. Add free_generation_demo(): one bs=1 generation that lets the model think
to completion and answer naturally on a single vignette, same vignette+schema as
the readout. evaluate() now prints both (DEMO A relabelled, DEMO B new) via
loguru when verbose, and returns them in result['demos'] so callers get the text
without return_per_row. Free think budget = min(2048, max(512, max_think*batch))
-- bs=1 frees batch memory, floored so even think=1 reasons, capped for big batches.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Per token-efficient-logging. evaluate(verbose=True) now: prints the first
row's full prompt+think+answer-slot trace (special tokens, promoted from DEBUG
to INFO, gated to first batch), the model-vs-human profile table, and a
one-line aux-stats dict. Set verbose=False inside sweeps.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Unifies forced-choice (nominal) and Likert (ordinal, expectation over the integer
distribution) on tinymfv's answer-token reader. Per scientist panel (docs/reviews/
sci_ma_*.md): canonicalize every frame's distribution to one forward orientation before
metrics+reducer (fixes the asymmetric nominal-reader/ordinal-reducer reflection), renormalize
p with pmass kept as canary, add ordinal |E-error| metric, keying applied only in the profile
reducer (proven orthogonal to framing, not a double-flip), cross-scale guard, negative-control
shuffle. 8 pure-function unit tests pass.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Chance-corrected, argmax-only companion to mean_nll: moves when the answer
flips, not when confidence shifts. 0 = base-rate guessing, so it exposes
majority-class models that top1_acc flatters. Same flip-informedness family
as steering-lite's surgical informedness, anchored on the human argmax here.
README also points at the paired training set moral_stories_foundations.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
When the model emits </think> in natural generation but the answer-slot
window detection fails, that's coherence collapse — the model "finished
thinking" without producing JSON. pmass=0.0 is the honest measurement
(no probability mass on allowed tokens at a non-existent slot) and lets
the coherence canary see the failure as a real signal rather than
propagating NaN through np.mean to crash c_scan. nll_json stays NaN
since no JSON was emitted to score.
Triggered by qwen3.6-27b nf4 + LoRA at c=1.0: 1/4 samples hit case (c)
and the NaN aborted c_scan instead of letting it walk down further.
Qwen3.6-27B nf4 + adapter at c=1.0 produced a non-finite raw logit at a
single generated step in 1/4 samples (others used forced-prefill path);
the natural-path F.log_softmax propagated NaN into mean_pmass_allowed,
crashing c_scan. Bound with nan_to_num(±1e4) — leaves argmax-finite rows
unchanged.
Drop legacy-cache-bug rationale from module + function docstrings; the design
stands on its own. Rename to match what the function does (no forking).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 1: batched generate with min_new_tokens=max_new_tokens so cache is uniform
length across the batch (no early stop at </think>). Phase 2: single batched
forced-suffix forward over that cache. Per-sample classification picks
gen.scores at the natural answer position (case a), forced logits (case b
interrupted), or NaN (case c emitted </think> but no answer).
Drops _slice_pkv_one + per-sample fork. The slice helper used layer.keys /
layer.values which crashes on Qwen3.5/3.6 LinearAttentionLayer (gated-delta-net
recurrent state has no .keys/.values). Uniform-length batched cache sidesteps
the cache surface entirely.
Bumps transformers>=5.7 for the Qwen3.5/3.6 gated-delta-net cached-forward
bugfix (resolves to 5.9.0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two paired changes the previous commit should have included.
skip_special_tokens kwarg on guided_rollout_forced_choice and evaluate()
threads into tok.decode for gen_text / gen_text_rev. Default False (return
the raw stream with </think>, chat markers, etc.) matches the "return all
the free things" principle. Callers who want stripped output strip
themselves.
emitted_close now uses a token-id match on gen_ids (`(gen_ids ==
think_end_id).any()`) instead of substring on the decoded text. On models
that mark </think> as a special token, the old substring check would
silently always return False when skip_special_tokens=True stripped it.
Qwen3 currently does NOT mark </think> as special so the bug is latent
there, but the fix is strictly more robust and decouples the detection
from the decode flag.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lets callers ask for N sampled think rollouts per direction instead of one
greedy trace. Per direction we Bayesian-model-average the answer logprobs
across the N samples (logsumexp_n lp - log N) before the fwd/rev average.
Raw per-sample [N, K] logprob matrices stay on the result as
lp_fwd_samples / lp_rev_samples so callers can re-aggregate (log-pooling,
majority vote, etc.).
gen_text and gen_text_rev are now always list[str] of length N (even at
N=1). think_tokens, think_tokens_rev, emitted_close, emitted_close_rev are
length-N lists. At N=1 the BMA is the identity and headline numbers match
the prior greedy path bit-for-bit.
Default max_think_tokens lowered 256 -> 64 for faster default eval (was
expensive overhead on small models that rarely emit </think> anyway).
README updated to match.
Phase 1.5 / Phase 2 already operated per-row, so they extend to B*N
expanded rows without change. Added an explicit assert that the HF
num_return_sequences expansion matches len(user_prompts) * n_samples.
Smoke-tested on Qwen3-0.6B: greedy N=1 matches BMA identity; N=4
temperature=0.7 returns [4, 7] sample matrices and finite pmass; guard
raises if n_samples>1 with temperature=0. evaluate() throughput log
extended to sum fwd+rev think tokens over all samples.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "negation" mention in pyproject + __init__ docstring was stale —
the actual second pass is internal fwd+rev enum-order debias inside
guided_rollout (position-bias cancellation), not a negation framing.
self_violate is not in Clifford 2015 classic (other-violation only).
Default `evaluate(..., conditions=...)` to ("other_violate",); callers
who want both can opt in explicitly. Halves walltime per eval.
CONDITIONS in data.py still lists both (other_violate, self_violate)
as available — the change is only the evaluate() default.
