diff --git a/README.md b/README.md index 77e4d86..22924d4 100644 --- a/README.md +++ b/README.md @@ -1,53 +1,18 @@ -# tinymfv (tiny moral/value eval for local LLMs) +# tinymfv -tinymfv reads answer-token logprobs from local chat LMs and turns them into moral/value profiles. -It is meant for fast steering experiments: prefill the answer slot, read the next-token -distribution, and compare base vs steered runs in nats instead of waiting for sampled answers to -flip. +tinymfv is a small set of quick value evals for local LLM steering work. It is sensitive to small answer-probability shifts, so you can see movement before sampled answers flip. -There are two instrument kinds: +It asks moral vignettes and survey questions, reads the model's answer probabilities, and compares the model profile to human responses. The main use case is simple: after you steer a model, did the intended values move, did nearby values move too, and does the result still look like a coherent answer? -- Nominal MFV vignettes: the answer is a foundation category. The profile is the mean probability - of care / fairness / loyalty / authority / sanctity / liberty / social. -- Ordinal questionnaires: the answer is a scale point 1..M. MFQ-2, Big Five, 16PF, and Humor Styles - all use the same reader and reduce to per-factor Likert profiles. +The plots are the point. Range plots show where the model sits relative to human societies. Culture maps show the base model and steered models on a PCA map of human response profiles. -The default nominal instrument is the Clifford moral-foundation vignettes (MFV): 132 scenarios, each -with a human distribution over foundations (Clifford et al. 2015). Three configs (`classic`, -`scifi`, `ai-actor`), two framings each (`other_violate`, `self_violate`). -[[HF dataset](https://huggingface.co/datasets/wassname/tiny-mfv)] +![MFQ-2 range plot: human society ranges beside the fairness steer path](docs/img/showcase/mfq2/range.png) -The bundled showcase steers Qwen3-4B with one activation vector, extracted once from -`wassname/moral_stories_foundations` fairness situations (mean activation difference, fairness vs a -balanced sample of the other foundations, situations only, no completions), iso-KL calibrated, then -read across every instrument at base / +c / -c. On MFQ-2 it selectively raises the equality factor: -equality is the only factor that rises at the gentlest +c, every other factor falls, a clean -directional move on the fairness-equality axis. On the MFV vignettes, though, the same vector moves -Care, not Fairness, so it is a care-leaning moral-salience direction that reads as fairness on the -questionnaire rather than a clean single-foundation isolator. Coherence holds across the sweep -(`frac_unscorable` 0, `mean_margin` ~10-12 nats), so the readouts below are not steering artifacts. +![MFV culture map: base and fairness-steered points against human countries](docs/img/showcase/mfv/map_pca_ipsative.png) -The same vector is read on all five bundled instruments. Range plots show the per-factor steer path -(red `+c` up, blue `-c` down) against human society ranges; maps are PCA culture maps with the c-sweep -trajectory. The 16PF map is omitted (16 axes do not lay out as a readable 2-D map; its range is kept). +![MFV range plot: Care moves most under the fairness vector](docs/img/showcase/mfv/range.png) -![MFQ-2 culture map: base and fairness-steered points against human societies](docs/img/showcase/mfq2/map_pca_ipsative.png) - -![MFQ-2 range plot: human society ranges beside the fairness steer path; equality rises at +c](docs/img/showcase/mfq2/range.png) - -![Big Five culture map with the c-sweep trajectory](docs/img/showcase/big5/map_pca_ipsative.png) - -![Big Five range: fairness steer path per trait against human society ranges](docs/img/showcase/big5/range.png) - -![Humor Styles culture map with the c-sweep trajectory](docs/img/showcase/humor_styles/map_pca_ipsative.png) - -![Humor Styles range: fairness steer path per style against human ranges](docs/img/showcase/humor_styles/range.png) - -![16PF range: fairness steer path across all 16 factors](docs/img/showcase/16pf/range.png) - -![MFV culture map with the c-sweep trajectory](docs/img/showcase/mfv/map_pca_ipsative.png) - -![MFV range: per-foundation emphasis; Care moves most under the fairness vector, Fairness barely](docs/img/showcase/mfv/range.png) +More maps are in `docs/img/showcase/` for MFQ-2, Big Five, 16PF, and Humor Styles. The plotting script now defaults to clean base / +C / -C maps; use `--show-sweep` when you want the diagnostic `c=-4..+4` trajectory. ## Install @@ -70,25 +35,42 @@ uv sync --extra maps --dev just smoke ``` -## Use it +## Datasets -Nominal MFV vignettes use `evaluate`: +| dataset | what it asks | model answer | human comparison | +|---|---|---|---| +| MFV classic | 132 Moral Foundations Vignettes from Clifford et al. | one of 7 foundations: Care, Fairness, Loyalty, Authority, Sanctity, Liberty, Social Norms | per-vignette human foundation labels | +| MFV scifi | the same MFV items rewritten as sci-fi scenarios | one of 7 foundations | inherited labels from the classic item | +| MFV ai-actor | the same MFV items rewritten so an AI system is the actor | one of 7 foundations | inherited labels from the classic item | +| MFQ-2 | 36 Moral Foundations Questionnaire items | 1-5 agreement | country foundation means, plus raw Atari et al. respondent data | +| Big Five | 50 personality items | 1-5 agreement | country factor means | +| 16PF | 162 personality items | 1-5 agreement | country factor means | +| Humor Styles | 32 humor-style items | 1-5 agreement | country style means, originally on a 1-7 scale | + +MFV is nominal: the answer is the category. The survey instruments are ordinal: the answer is a scale point. + +Each MFV item is asked in two perspectives, `other_violate` and `self_violate`. Each survey item is asked three ways, forward, scale-inverted, and content-negated. tinymfv canonicalizes these frames before averaging, so the profile is less tied to one wording. + +## API + +Run MFV vignettes with `evaluate`: ```python from transformers import AutoModelForCausalLM, AutoTokenizer -from tinymfv import load_vignettes, evaluate +from tinymfv import evaluate, load_vignettes tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B") model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B").cuda() -vignettes = load_vignettes("classic") # list[dict], one per scenario +vignettes = load_vignettes("classic") # "classic", "scifi", "ai-actor", or "all" report = evaluate(model, tok, vignettes=vignettes, return_per_row=True) -print(report["mean_pmass_allowed"]) -print(report["per_row"][0]["score"]) # logprob score per foundation, nats -print(report["profile"]) # mean p[foundation], easier to read + +print(report["profile"]) # mean probability per foundation +print(report["mean_pmass_allowed"]) # probability mass on valid answer tokens +print(report["per_row"][0]["score"]) # foundation logprobs, in nats ``` -Ordinal questionnaires use `administer`: +Run survey instruments with `administer`: ```python from transformers import AutoModelForCausalLM, AutoTokenizer @@ -97,165 +79,60 @@ from tinymfv import administer, get_instrument tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B") model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B").cuda() -instr = get_instrument("mfq2") # "mfq2", "big5", "16pf", or "humor_styles" +instr = get_instrument("mfq2") # "mfq2", "big5", "16pf", or "humor_styles" report = administer(model, tok, instr) + print(report["dimensions"]) -print(report["per_item_frame"][0]["lp"]) # raw logprobs for the answer tokens -print(report["profile_E"]) # human-comparable factor means -print(report["profile_C"]) # steer-sensitive logit contrast per factor -print(report["mean_pmass_allowed"]) +print(report["profile_E"]) # human-comparable scale means +print(report["profile_C"]) # steering-sensitive log contrast +print(report["per_item_frame"][0]["lp"]) # raw answer-token logprobs ``` -From a checkout, the small functional run is: +## Metrics -```bash -just smoke -just eval Qwen/Qwen3-0.6B classic -``` +There are two metric families. -The plotting helpers live in `tinymfv.maps`; `scripts/plot_steer_showcase.py` shows the current -map/range pipeline for steering-lite outputs. +Human-comparison metrics are bounded and easy to plot against survey data. -## Reader math +For MFV, the profile is the mean probability of each foundation: -For an answer set $A = \{a_1,\ldots,a_K\}$, tinymfv gathers the full-vocab logprobs at the answer -tokens: +$$p_f = \mathbb{E}_i\,P(f \mid i)$$ -$$\ell_k = \log P(a_k \mid \text{prompt}, \text{think trace}, \text{assistant prefill})$$ +For surveys, `profile_E` is the expected scale score, reverse-keyed where needed: -The raw gathered vector $\ell$ is the primitive. Everything else is a pure readout: +$$E_i = \sum_{k=1}^{M} k p_{i,k}$$ -$$\mathrm{pmass\_allowed} = \sum_{k=1}^{K} \exp(\ell_k)$$ +Steering metrics are logprob based. They are more sensitive because they move before the sampled answer flips. -$$p_k = \frac{\exp(\ell_k)}{\mathrm{pmass\_allowed}}$$ +At the answer slot, tinymfv gathers the logprobs for the allowed answers: -`pmass_allowed` is answer-format coherence: mass on valid answer tokens at the answer slot. The -slot is force-prefilled, so the scaffold primes a valid token and `pmass_allowed` is pinned high -under the forced-choice path. Treat it as a format floor, not a sensitive coherence gate: a steered -run that has broken the model can still score `pmass_allowed` near 1.0. The two signals that carry -coherence instead are `frac_unscorable` (rows with no scorable slot, a self-closed think with no -answer or a blown-up forward; near 0 once the end-of-answer tokens are suppressed over the budget) -and `nll_prefill` below (does the scaffold still fit the model). pmass does not say which valid -answer is right. +$$\ell_k = \log P(a_k \mid \mathrm{prompt}, \mathrm{think}, \mathrm{prefill})$$ -`nll_prefill` measures whether the forced assistant prefill itself fits the model: +For surveys, `profile_C` is the steering-sensitive direction score. It weights high agreement tokens positive and low agreement tokens negative, using logprobs instead of bounded survey means: -$$\mathrm{nll\_prefill} = -\frac{1}{J}\sum_{j=1}^{J} \log P(u_j \mid \text{context}, u_{`) are suppressed over the budget, so it - cannot self-close mid-trace into an unscorable state; every read is taken at the same forced slot, - comparable across base and steered, instead of the old readout collapse once the model self-closed. -- Two modes. dev (N=1, greedy, 64 think) is fast and granular, the default. full (N=4 sampled traces - per ordering, Bayesian-model-averaged, high think) adds sampling-variance uncertainty and SI. - -## Instruments - -The reader is answer-space-agnostic: it gathers logprobs over answer tokens at a prefilled slot -(`src/tinymfv/instrument.py`). Ordinal instruments read a 1..M Likert point and reduce to a keyed -expected score `E`, the rank-centered logit contrast `C`, `logodds_agree`, entropy, and -`pmass_allowed`; the nominal MFV vignettes read a foundation category and reduce to the log-prob over -foundations (`dlogit`). - -"Ways asked" is the per-item debias: ordinal items are presented in three reworded framings -(forward / scale-inverted / content-negated, canonicalized and averaged, which cancels acquiescence); -MFV runs two option-order passes (forward / reversed enumeration, which cancels position bias). - -| instrument | items | ways asked | measure | per-respondent | per-country | per-item human | source | -| :--- | ---: | :--- | :--- | :--- | ---: | :--- | :--- | -| MFQ-2 (Moral Foundations) | 36 | 3 reworded framings | log-odds + contrast `C`, 1-5 Likert | yes (raw) | 19 | no | Atari et al. 2023 | -| Big Five | 50 | 3 reworded framings | log-odds + contrast `C`, 1-5 Likert | no | 24 | no | BFI (country source not recorded in-repo) | -| 16PF | 162 | 3 reworded framings | log-odds + contrast `C`, 1-5 Likert | no | 34 | no | Cattell 16PF (country source not recorded) | -| Humor Styles | 32 | 3 reworded framings | log-odds + contrast `C`, model 1-5 / human 1-7 | no | 28 | no | Martin et al. 2003 | -| MFV (Moral Foundations Vignettes) | 132 | 2 option-order passes | log-prob over 7-way categorical (`dlogit`) | no | 5 | yes (per-vignette) | Clifford et al. 2015; norms JimenezLeal2025 + Yamada2025 | - -Items is the unique-item count; ordinal items are each scored x3 framings. Per-respondent = raw -individual human data is bundled (MFQ-2 ships Atari et al. Study 2 respondents, used for the SPLOM and -the PCA basis); per-country = number of societies with published mean+sd; per-item human = per-question -human distribution (only the MFV vignettes carry Clifford's per-vignette ratings). - -The bundled public showcase figures (per instrument, identical layout) are: - -- `docs/img/showcase//map_pca_ipsative.png`: culture map, model base and steer poles against - human societies. -- `docs/img/showcase//range.png`: per-factor human ranges beside the model steer path. +If it drops, the model is leaking probability into invalid answers. If `nll_prefill` rises, the forced answer scaffold itself no longer fits the model well. ## Scope -A fast sensitive eval for small steering interventions on local models, not a full moral-reasoning -evaluation. For behaviour-heavy evals see -[machiavelli](https://huggingface.co/datasets/wassname/machiavelli), -[AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas), -[ethics_expression_preferences](https://huggingface.co/datasets/wassname/ethics_expression_preferences). +tinymfv is for fast paired steering comparisons, not full moral reasoning evaluation. It is useful when you want to compare base, positive-steer, and negative-steer runs against the same human reference plots. -Used in [steering-lite](https://github.com/wassname/steering-lite), -[lora-lite](https://github.com/wassname/lora-lite), -[w2schar-mini](https://github.com/wassname/w2schar-mini). +For behavior-heavy moral evals, see [machiavelli](https://huggingface.co/datasets/wassname/machiavelli), [AIRiskDilemmas](https://huggingface.co/datasets/kellycyy/AIRiskDilemmas), and [ethics_expression_preferences](https://huggingface.co/datasets/wassname/ethics_expression_preferences). + +Used in [steering-lite](https://github.com/wassname/steering-lite), [lora-lite](https://github.com/wassname/lora-lite), and [w2schar-mini](https://github.com/wassname/w2schar-mini). ## Citation diff --git a/scripts/plot_steer_showcase.py b/scripts/plot_steer_showcase.py index f367bb0..eba2352 100644 --- a/scripts/plot_steer_showcase.py +++ b/scripts/plot_steer_showcase.py @@ -110,7 +110,7 @@ def read_profiles(run_dir: Path, name: str, dims: list[str], value_col: str = "m return {c: np.array([d[f] for f in dims]) for c, d in by_c.items()}, pmass -def plot_ordinal(run_dir: Path, out: Path, name: str, vec_label: str, C: float) -> list[Path]: +def plot_ordinal(run_dir: Path, out: Path, name: str, vec_label: str, C: float, *, show_sweep: bool = False) -> list[Path]: instr = get_instrument(name) dims = instr.dimensions prof_c, pmass = read_profiles(run_dir, name, dims) @@ -133,12 +133,14 @@ def plot_ordinal(run_dir: Path, out: Path, name: str, vec_label: str, C: float) respondents, haze = T.maps.respondent_profiles(dims, instr.scale_max), None else: respondents, haze = None, human_haze(instr) - # trajectory overlay only when the run swept more than the 3-point base/+-C (else the arrows suffice). + # Public showcase maps default to the clean base/+-C anchors. The full -N..+N sweep is useful + # for diagnosis, but it clutters the README and is often mistaken for incoherent random dots. + # Pass --show-sweep when debugging how stronger coefficients leave the human map. # Coherence gate is RELATIVE: keep a c only if its pmass stays within 95% of the base (c=0) pmass; # below that the readout has degraded enough that the profile is not comparable, so drop it entirely. base_pm = pmass[0.0] coh_cs = [c for c in cs if pmass[c] >= 0.95 * base_pm] - traj = {c: _frac(prof_c[c], instr.scale_max) for c in coh_cs} if len(coh_cs) > 3 else None + traj = {c: _frac(prof_c[c], instr.scale_max) for c in coh_cs} if show_sweep and len(coh_cs) > 3 else None traj_inco = None # excluded (not drawn hollow) per the 95%-of-base coherence gate figm = T.maps.plot_ipsative_pca(instr, dims, countries, Mfrac, _frac(base, instr.scale_max), _frac(pos, instr.scale_max), @@ -236,6 +238,8 @@ def main() -> None: ap = argparse.ArgumentParser() ap.add_argument("--run-dir", type=Path, required=True) ap.add_argument("--out", type=Path, default=Path("docs/img/showcase")) + ap.add_argument("--show-sweep", action="store_true", + help="draw the full coherence-gated c sweep on ordinal maps") args = ap.parse_args() summary = json.loads((args.run_dir / "summary.json").read_text()) C = float(summary["calibrated_C"]) @@ -246,7 +250,8 @@ def main() -> None: written: list[str] = [] for name in ORDINAL: if (args.run_dir / f"{name}_profiles.csv").exists(): - written += [str(p) for p in plot_ordinal(args.run_dir, args.out, name, vec_label, C)] + written += [str(p) for p in plot_ordinal(args.run_dir, args.out, name, vec_label, C, + show_sweep=args.show_sweep)] if (args.run_dir / "mfv.json").exists(): written.append(str(plot_mfv_map(args.run_dir, args.out, vec_label, C))) # shared ipsative map (z-space) written.append(str(plot_mfv_range(args.run_dir, args.out, vec_label, C))) # shared range (z-space)