Document WVS panel exclusions

Co-Authored-By: PI[gpt-5.6-terra] <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-09-17 04:33:46 +08:00
co-authored by PI[gpt-5.6-terra]
parent 8db3d3daa9
commit 2e2914db0e
8 changed files with 4937 additions and 4479 deletions
+4 -2
View File
@@ -5,9 +5,11 @@ moralmaps is a small set of fast value evals for LLM steering work. It asks surv
## Are models moral aliens?
![Inglehart-Welzel culture map with 17 frontier models placed among 90 human societies, scored by rated sampling. Horizontal axis: self-expression on the left, survival on the right. Vertical axis: secular-rational at the top, traditional at the bottom. Coloured outlines mark the West, East Asia, Latin America and African-Islamic zones. Every model star sits in the upper left, and gpt-5.5, grok-4.3, gemini-2.5-pro and deepseek-v4-pro sit above the West outline entirely, higher than Sweden. The models run 0.53 to 0.76 on secular-rational against 0.06 to 0.70 for the societies, but none passes Iceland on self-expression. The push is vertical, not sideways.](docs/img/wvs/wvs_map_iw.png)
![Inglehart-Welzel culture map with 64 frontier-model coordinates among human societies, scored by rated sampling. Horizontal axis: self-expression on the left, survival on the right. Vertical axis: secular-rational at the top, traditional at the bottom. Coloured outlines mark the West, East Asia, Latin America and African-Islamic zones. The upper-left model cluster is crowded; use the CI table for exact coordinates and sample evidence.](docs/img/wvs/wvs_map_iw.png)
One interesting thing we can do with this repo is put AI models through human psychological and anthropological surveys. Are they like us? Start with the World Values Survey, the standard culture map of the world: since 1981 it has asked people in about ninety countries the same questions, and two axes drawn from it sort societies by how traditional or secular they are and how much they weigh survival over self-expression. We put seventeen frontier models through the same questions (`scripts/wvs_map.py` makes this map).
One interesting thing we can do with this repo is put AI models through human psychological and anthropological surveys. Are they like us? Start with the World Values Survey, the standard culture map of the world: since 1981 it has asked people in about ninety countries the same questions, and two axes drawn from it sort societies by how traditional or secular they are and how much they weigh survival over self-expression. The map combines 17 recovered rounded historical coordinates with 48 newly completed rated panels (`scripts/wvs_map.py`). One name overlaps, so it displays 64 coordinates. The newly measured points trace to the [request ledger](slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl); incomplete attempts are not plotted.
The new panel adds Fable 5.1, GPT-6 Astra, DeepSeek V4.1 Flash, Kimi K3, Muse Spark 1.3, Inkling, GLM 5.3, Gemini 3.7 Flash, Grok 4.5, GPT-5.6 Sol, and currently hosted Qwen releases. [The catalog matrix](slop/research/wvs/20260916_openrouter/execution_matrix.md) records exact IDs, dates, prices and reasoning/schema eligibility. It does not establish that capability or release order causes an axis movement: comparable Qwen direct-instruct points vary across both axes, while Coder and VL models are separately named specialized variants. GLM 5.3 Flash and Grok 4.5 remain excluded because their retained runs are 143/144; no family trend is inferred from them.
Every model sits in the top-left, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies. The push is almost all vertical. Measured in the standard deviations of the 29 Western societies, every model is more secular-rational than the average one, from +0.5 to +2.9 sigma, while on self-expression they land between -0.7 and +1.2 sigma, which is ordinary. So they are not so much an ultra Silicon Valley point as a place north of the map that no society occupies.
+2 -2
View File
@@ -15,9 +15,9 @@ Regenerate everything with the two commands at the bottom.
## No country thinks like a chatbot
*17 frontier LLMs placed among ~90 human societies on the World Values Survey map*
*64 model coordinates among ~90 human societies on the World Values Survey map*
![WVS culture map: 17 LLMs among ~90 societies on the Inglehart-Welzel axes](wvs/wvs_map_iw.png)
![WVS culture map: 64 model coordinates among ~90 societies on the Inglehart-Welzel axes. The upper-left model cluster is crowded; use the linked CI table for exact values.](wvs/wvs_map_iw.png)
Grey dots are human societies, coloured stars are language models tinted by lab family (warm for
Chinese labs, cool for Western). Left to right runs from survival values to self-expression; bottom to
Binary file not shown.

Before

Width:  |  Height:  |  Size: 408 KiB

After

Width:  |  Height:  |  Size: 388 KiB

+2095 -4469
View File
File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 514 KiB

After

Width:  |  Height:  |  Size: 438 KiB

+9 -6
View File
@@ -386,18 +386,21 @@ def main() -> None:
# Drop the " (rated)" readout tag from the on-map labels (the cache/CI-table keep it) -- the map is
# crowded and every model here is rated, so the tag adds nothing.
plot_models = {k.replace(" (rated)", ""): v for k, v in models.items()}
# Too many model names to label them all. Plot every star (colour = family) but LABEL only the
# latest model per family (highest version number), and drop the redundant "claude-" so the flagship
# reads "opus-4.8". Colour + legend carry the unlabelled siblings.
# Too many model names to label them all. Plot every star (colour = family) but label only the
# latest catalogued release in each family. Numeric version strings do not establish release order.
fams: dict[str, list[str]] = {}
for k in plot_models:
family = maps.model_family(k)
if family is None:
raise ValueError(f"model has no explicit family: {k}")
fams.setdefault(family, []).append(k)
def _ver(k: str) -> list[float]:
return [float(n) for n in re.findall(r"\d+(?:\.\d+)?", k)]
model_labels = {max(ks, key=_ver): max(ks, key=_ver).replace("claude-", "") for ks in fams.values()}
metadata = json.loads(Path("docs/img/wvs/wvs_model_metadata.json").read_text())["models"]
model_labels: dict[str, str] = {}
for family, names in fams.items():
catalogued = [name for name in names if name in metadata]
if catalogued:
latest = max(catalogued, key=lambda name: metadata[name]["created"])
model_labels[latest] = latest.replace("claude-", "")
# Poles in NATURAL data order (x_neg, x_pos, y_neg, y_pos): raw X is high on Self-expression, raw Y
# high on Secular-Rational. plot_value_map's orient_geographic then flips X so the cultural West
# lands in the west (Self-expression left) and confirms African-Islamic sits south -- the same
File diff suppressed because it is too large Load Diff
@@ -2,6 +2,10 @@
Checked from the saved OpenRouter catalog on 2026-09-16. Protocol IDs hash all rendered WVS prompts and settings. `response_format` is only requested where the catalog advertises it. A mandatory model uses its least listed effort. Entries with mandatory reasoning but no supported-effort list are excluded rather than guessing a setting. These are candidate runs, not completed points.
## Actual mandatory-reasoning batch
The executed mandatory batch used one shared setting, `reasoning={"effort":"low"}`, with strict JSON schema and concurrency 1 for `anthropic/claude-fable-5.1`, `openai/gpt-6-astra`, `meta/muse-spark-1.3`, `z-ai/glm-5.3`, `z-ai/glm-5.3-flash`, `google/gemini-3.7-flash`, and `x-ai/grok-4.5`. Therefore the candidate hash for Muse below, which uses the catalog minimum `minimal`, is not the executed protocol. The append-only request ledger is the source of truth for actual run ID, protocol hash, raw request settings, usage and errors. -- PI[gpt-5.6-terra]
| exact ID | catalog name | created UTC | input USD/M | output USD/M | reasoning | schema | protocol ID |
|---|---|---:|---:|---:|---|---|---|
| `anthropic/claude-fable-5.1` | Anthropic: Claude Fable 5.1 | 2026-09-01 | 10 | 50 | mandatory low | True | `4230ccaa1e160c793a906aef6b786d735d9fdbf71c3333691b0eba17dcf33928` |
@@ -0,0 +1,15 @@
# Fresh-eyes review: WVS static map
Artifact reviewed: `docs/img/wvs/wvs_map_iw.png`.
Reviewer: `reviewer-anthropic`, read-only image review, 2026-09-16. This is an observation of the rendered image, not an audit of data or code.
> Overall readability: moderate. The axes, quadrant labels, title, and cultural-zone hulls are clear. The upper-left is badly cluttered.
> Orange overload / color collision. The many orange stars share the exact hue of the Latin America hull.
> Label-to-marker ambiguity. Model labels float without leader lines and several are far from any same-colored star.
> `deepseek-v4.1-flash` label overlaps `Great Britain` label. `mistral-large-2512` and `llama-4-maverick` labels stack tightly with `Sweden`.
The latest rendered image now says `64 models`, so the reviewer's earlier caption observation (`17 models`) has been corrected. The color collision, crowded upper-left, and label ambiguity remain visible in the latest PNG. The static geometry is interpretable but this does not pass the plan's readability discriminator for a published interactive page. No Pages deployment was attempted. -- PI[gpt-5.6-terra]