mirror of
https://github.com/wassname/moral-maps.git
synced 2026-09-11 12:20:38 +08:00
Put the outlier table in the README and number-bearing alt text on the plots
Every plot's numbers lived only in the pixels, so a text scraper or an LLM reading the repo saw nothing. Also drops the "more self-expressive than almost any country" claim: none of the 17 passes Iceland, and against the West they average about zero on that axis. The displacement is vertical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f6c22aca30
commit
c762fce103
@@ -5,12 +5,36 @@ moralmaps is a small set of fast value evals for LLM steering work. It asks surv
|
||||
## Are models moral aliens?
|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
One interesting thing we can do with this repo is put AI models through human psychological and anthropological surveys. Are they like us? Start with the World Values Survey, the standard culture map of the world: since 1981 it has asked people in about ninety countries the same questions, and two axes drawn from it sort societies by how traditional or secular they are and how much they weigh survival over self-expression. We put seventeen frontier models through the same questions (`scripts/wvs_map.py` makes this map).
|
||||
|
||||
|
||||
Every model sits in the top-left: more secular and more self-expressive than almost any country on earth, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies, an ultra Silicon Valley cultural point. This map is measured differently from everything else on the page. These frontier models are closed APIs with no answer probabilities to read, so each is scored by rated sampling (rate every option one to five, twelve times, with the option order shuffled; `scripts/wvs_map.py`), and the human positions are approximated from the GlobalOpinionQA question set (axis construction in `src/moralmaps/iw_axes.py`). The steering plots below instead follow one open model we can push, Qwen3-4B.
|
||||
Every model sits in the top-left, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies. The push is almost all vertical. Measured in the standard deviations of the 29 Western societies, every model is more secular-rational than the average one, from +0.5 to +2.9 sigma, while on self-expression they land between -0.7 and +1.2 sigma, which is ordinary. So they are not so much an ultra Silicon Valley point as a place north of the map that no society occupies.
|
||||
|
||||
| model | z self-expr | z secular | Mahalanobis |
|
||||
|:------|------------:|----------:|------------:|
|
||||
| gpt-5.5 | -0.33 | +2.93 | +4.71 |
|
||||
| deepseek-v4-pro | +0.54 | +2.61 | +3.32 |
|
||||
| grok-4.3 | -0.19 | +2.61 | +4.08 |
|
||||
| gemini-2.5-pro | +0.01 | +2.29 | +3.38 |
|
||||
| qwen3.7-max | -0.66 | +2.18 | +4.01 |
|
||||
| gpt-5.4 | -0.13 | +2.07 | +3.21 |
|
||||
| gpt-5.3-chat | +0.21 | +1.96 | +2.69 |
|
||||
| gemma-4-31b-it | -0.73 | +1.86 | +3.62 |
|
||||
| deepseek-v4-flash | +0.68 | +1.64 | +1.83 |
|
||||
| llama-4-maverick | +1.14 | +1.64 | +1.65 |
|
||||
| mistral-large-2512 | +1.21 | +1.64 | +1.64 |
|
||||
| claude-opus-4.6 | +1.08 | +1.54 | +1.54 |
|
||||
| grok-4.20 | +0.81 | +1.43 | +1.47 |
|
||||
| claude-opus-4.7 | +0.81 | +1.21 | +1.22 |
|
||||
| gemma-3-27b-it | +0.88 | +1.21 | +1.21 |
|
||||
| claude-opus-4.8 | +0.94 | +1.00 | +1.04 |
|
||||
| llama-4-scout | +1.01 | +0.46 | +1.09 |
|
||||
|
||||
Distance from the centroid of the 29 Western societies, in that cluster's own SDs (`scripts/wvs_outlier_table.py`). The per-axis z says which way and how far; the Mahalanobis column says how odd the placement is overall, and it uses the cluster's covariance, so it exceeds both z values for a model like gpt-5.5 that sits off the West's diagonal rather than along it. The same table against the other four zones is in [`wvs_model_outlier_sd.md`](docs/img/wvs/wvs_model_outlier_sd.md).
|
||||
|
||||
This map is measured differently from everything else on the page. These frontier models are closed APIs with no answer probabilities to read, so each is scored by rated sampling (rate every option one to five, twelve times, with the option order shuffled; `scripts/wvs_map.py`), and the human positions are approximated from the GlobalOpinionQA question set (axis construction in `src/moralmaps/iw_axes.py`). The steering plots below instead follow one open model we can push, Qwen3-4B.
|
||||
|
||||
The Economist ran a similar, nicely-made map in June 2026 ([briefing, archived](https://web.archive.org/web/20260630075107/https://www.economist.com/briefing/2026/06/25/ai-models-values-are-very-different-from-most-peoples)), putting 25 frontier models through the same Inglehart-Welzel axes. Their figure shows a surprising amount of scatter between model families: same-lab models can land in opposite corners (DeepSeek R1 sits up in the secular corner beside GPT-4o, while DeepSeek V4 Flash sits far off toward the traditional societies). moralmaps reruns that idea with more sensitive, graded readings (rate every option one to five with the order shuffled, rather than a handful of near-greedy answers) and a 95% confidence interval per model ([`wvs_model_ci.md`](docs/img/wvs/wvs_model_ci.md)), so we can tell how much of that scatter is real signal and how much is measurement noise.
|
||||
|
||||
@@ -31,15 +55,15 @@ Models have generally been trained to follow the instructions of the company tha
|
||||
Below are the ("quadrant") maps. Each has two named axes borrowed from psychology papers built from the survey, the human societies are drawn as cultural regions, and the model as a black dot with a coloured path showing where steering takes it. Steering here is activation steering, not prompting: a vector built by [steering-lite](https://github.com/wassname/steering-lite) from contrastive persona pairs and added to the model's hidden state at inference, toward the authority-respecting side (red, more Authority) or away from it (blue, less), without retraining. Every map keeps one orientation, the cultural West to the west and the global South to the south, so they all read the same way.
|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
Moral-foundations theory (Jonathan Haidt's) holds that our moral sense runs on a few basic concerns: caring for others, fairness, loyalty to the group, respect for authority, and a sense of the sacred. The MFQ-2 survey (Moral Foundations Questionnaire) scores a person, or a model, on each. On this map, left to right runs from an individual-first morality (care, equality) to a group-first one (loyalty, authority, purity); bottom to top splits fairness into equal-shares versus earned-shares. The base model sits in the Western, individual-first corner, and pushing it toward Authority walks it clear across to the group-first corner shared by the African-Islamic and East-Asian societies.
|
||||
|
||||

|
||||

|
||||
|
||||
Big Five personality collapses to two broad traits: how outgoing and open a person is (reserved to exploratory, left to right) and how even-keeled they are (volatile to stable, bottom to top). The Authority push barely moves the base model here, which is the point: it shifts values, not personality.
|
||||
|
||||

|
||||

|
||||
|
||||
Humor shows little variation on the map (although the range plots below show some nuance). On its axes (warm, healthy humor versus put-down humor; joking at yourself versus at others) the human regions overlap almost completely: humor style does not sort societies the way values do. Worth knowing a survey can't tell societies apart at all before reading anything into a steer on it.
|
||||
|
||||
@@ -47,15 +71,15 @@ Humor shows little variation on the map (although the range plots below show som
|
||||
|
||||
A range plot takes one survey at a time, factor by factor: the spread of human societies is a grey strip, their middle a black line, and the steer a red-to-blue sweep, so even a small model move stays visible against the whole human range.
|
||||
|
||||

|
||||

|
||||
|
||||
MFV (moral-foundation vignettes, the repo's namesake) hands the model a short story about someone breaking a moral rule and asks which kind of wrong it is: cruelty, cheating, betrayal, defiance of authority, or defiling the sacred. Pushed toward Authority, the model does what steering should: it flags the authority violations far more often and the others less. The grey dot per foundation is a pooled human reference; the base model already flags authority violations well above the pooled human rate, and the steer pushes it further still. That human dot is pooled on purpose: MFV country norms fail cross-country measurement invariance ([Jimenez-Leal et al. 2025](https://doi.org/10.1525/collabra.128178)) and are stitched from five different studies, so MFV gets no culture map here, only this range against one pooled reference (details in [`src/moralmaps/data/human/MFV_country_norms_NOTE.md`](src/moralmaps/data/human/MFV_country_norms_NOTE.md)).
|
||||
|
||||

|
||||

|
||||
|
||||

|
||||

|
||||
|
||||

|
||||

|
||||
|
||||
The surveys echo their maps: MFQ-2's binding foundations (loyalty, authority, purity) climb under the steer, while Big Five and humor stay flat.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user