The default ordinal readout was the expected Likert score E = sum k*p_k, which is
insensitive to steering: dE/dl_j = p_j(j-E) vanishes when the model answers confidently
(peaked at the mode), so a steer that reallocates the tails barely moves E. read.py threw
away the raw logprobs after renormalizing, so nothing downstream could recover the signal.
- read.py keeps the raw lp_gather (the primitive) + the think traces on every row.
- readouts.py: pure functions of lp -- expected_score E (human-comparable), logit_contrast
C = sum (k-mid)*lp_k (primary steer signal: dC/dl_j = w_j, no p_j suppression, normalizer-
invariant, dC = w.dl exactly), agree_logodds LO (readable 2-bin direction), entropy.
- per_item_categorical also frame-averages the logprobs (exact for the linear contrast).
- administer returns profile_C alongside profile_E, per-item E/C/LO/entropy with bootstrap
CIs for both, and the raw per-(item,frame) rows with lp + think for downstream reconstruction.
Unit check: on a peaked-at-4 dist under a small disagree steer, dE=-0.11 but dC=-1.80
(=w.dl exactly) and dLO=-0.60; C identical on raw logits vs renormalized logprobs.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
administer()/read_items() now route through _rollout_natural_or_forced (the
nominal MFV core) instead of a think=0 single forward, so an activation steer
accrues over the think trace before the prefilled answer slot is read (spec
moral_aliens_engine.md, resolved decision: ordinal needs a think budget). The
only per-instrument difference is the answer-token set + the downstream reducer.
force_only on the shared core: the ordinal "(" prefill is one common char, so
natural-emission detection would match it by chance in the think trace and read
logits mid-think; surveys always force-read the answer slot. Nominal path keeps
natural emission (force_only defaults False). max_think_tokens floor is 1.
Smoke (tiny-random): ordinal + nominal both run; force-only demo reads the
forced ( slot.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Whole-library review (deepseek-v4-pro) flagged tinymfv as not-yet-ready as a shared
dep. Fixes for the parts I agreed with:
- lazy `maps` import via module __getattr__ so `import tinymfv` stays headless/fast
(no forced matplotlib) for numeric-only consumers; `tinymfv.maps.*` still works.
- trim __all__ to the front door (entrypoints + types + data api); plumbing stays
importable but out of `import *`.
- delete dead code: reduce_nominal + REDUCERS (evaluate folds its profile inline),
expected_value, HF_REPO, ROOT, _DEFAULT_FORCED_HINT.
- type administer's return as a TypedDict (AdministerResult/ItemRow/ItemFrameRow) so
the schema is documented + checkable without reading source; still a plain dict at
runtime (zero consumer churn).
- maps.plot_ipsative_pca: parametrize the legend labels (defaults preserve output)
and rename hon/dis -> pos/neg so a non-honesty steer gets a correct legend.
- drop 'canary' jargon and panel/review-# archaeology from comments.
Verified: `import tinymfv` no longer loads matplotlib; lazy maps still resolves;
experiment mfq2 smoke green through the typed administer.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
deepseek-v4-pro review of the admin->tinymfv port (algebra proven correct, diff
8.25e-08 = float noise). Closes its flagged fail-fast gaps:
- per_item_categorical asserts uniform frame count per item (else the per-item
average would silently reweight a factor)
- reduce_ordinal asserts dimension is not None (no phantom-factor pooling)
- administer asserts every ordinal item carries meta['task'] (else build_prompt
would silently drop the response-scale legend and the profile would be junk)
Also commits maps.py + viz deps from the stage-3 port.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Port the answer-token survey readout from the weight_steer_honesty experiment
(mft_honesty.admin) onto the instrument.py canonicalize-at-reader design:
- read.py: generalized answer-token reader (any answer_space + prefill)
- administer.py: read all frames -> per_item_categorical -> reduce_ordinal -> profile
- instruments.py: build MFQ-2/Big5/16PF/HSQ Instruments from bundled survey JSONs
- instrument.py: add display + human_csv fields for the map layer
- data/: survey JSONs + human country CSVs (lean: no raw survey responses)
Parity: experiment's parity_administer_check.py shows max per-foundation diff
8.25e-08 vs admin.administer on the tiny model (same function of same logits).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The "negation" mention in pyproject + __init__ docstring was stale —
the actual second pass is internal fwd+rev enum-order debias inside
guided_rollout (position-bias cancellation), not a negation framing.
self_violate is not in Clifford 2015 classic (other-violation only).
Default `evaluate(..., conditions=...)` to ("other_violate",); callers
who want both can opt in explicitly. Halves walltime per eval.
CONDITIONS in data.py still lists both (other_violate, self_violate)
as available — the change is only the evaluate() default.
Was emitting `logger.warning("pmass=0.XX<0.9 — top-5: ...")` per-row, which
spammed the log heavily during heavy-steering eval (many rows go OOD at once).
Now collects all low-pmass rows in the batch and emits one summary line with
the worst-case top-5, e.g.:
pmass<0.9 on 7/16 rows in this batch; worst=0.412 top-5: '1'=0.40, ...
Same diagnostic signal, ~16× fewer log lines per batch.
