mirror of
https://github.com/wassname/moral-maps.git
synced 2026-09-21 13:10:52 +08:00
Document WVS panel exclusions
Co-Authored-By: PI[gpt-5.6-terra] <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
co-authored by
PI[gpt-5.6-terra]
parent
8db3d3daa9
commit
2e2914db0e
@@ -5,9 +5,11 @@ moralmaps is a small set of fast value evals for LLM steering work. It asks surv
|
||||
## Are models moral aliens?
|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
One interesting thing we can do with this repo is put AI models through human psychological and anthropological surveys. Are they like us? Start with the World Values Survey, the standard culture map of the world: since 1981 it has asked people in about ninety countries the same questions, and two axes drawn from it sort societies by how traditional or secular they are and how much they weigh survival over self-expression. We put seventeen frontier models through the same questions (`scripts/wvs_map.py` makes this map).
|
||||
One interesting thing we can do with this repo is put AI models through human psychological and anthropological surveys. Are they like us? Start with the World Values Survey, the standard culture map of the world: since 1981 it has asked people in about ninety countries the same questions, and two axes drawn from it sort societies by how traditional or secular they are and how much they weigh survival over self-expression. The map combines 17 recovered rounded historical coordinates with 48 newly completed rated panels (`scripts/wvs_map.py`). One name overlaps, so it displays 64 coordinates. The newly measured points trace to the [request ledger](slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl); incomplete attempts are not plotted.
|
||||
|
||||
The new panel adds Fable 5.1, GPT-6 Astra, DeepSeek V4.1 Flash, Kimi K3, Muse Spark 1.3, Inkling, GLM 5.3, Gemini 3.7 Flash, Grok 4.5, GPT-5.6 Sol, and currently hosted Qwen releases. [The catalog matrix](slop/research/wvs/20260916_openrouter/execution_matrix.md) records exact IDs, dates, prices and reasoning/schema eligibility. It does not establish that capability or release order causes an axis movement: comparable Qwen direct-instruct points vary across both axes, while Coder and VL models are separately named specialized variants. GLM 5.3 Flash and Grok 4.5 remain excluded because their retained runs are 143/144; no family trend is inferred from them.
|
||||
|
||||
|
||||
Every model sits in the top-left, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies. The push is almost all vertical. Measured in the standard deviations of the 29 Western societies, every model is more secular-rational than the average one, from +0.5 to +2.9 sigma, while on self-expression they land between -0.7 and +1.2 sigma, which is ordinary. So they are not so much an ultra Silicon Valley point as a place north of the map that no society occupies.
|
||||
|
||||
+2
-2
@@ -15,9 +15,9 @@ Regenerate everything with the two commands at the bottom.
|
||||
|
||||
## No country thinks like a chatbot
|
||||
|
||||
*17 frontier LLMs placed among ~90 human societies on the World Values Survey map*
|
||||
*64 model coordinates among ~90 human societies on the World Values Survey map*
|
||||
|
||||

|
||||

|
||||
|
||||
Grey dots are human societies, coloured stars are language models tinted by lab family (warm for
|
||||
Chinese labs, cool for Western). Left to right runs from survival values to self-expression; bottom to
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 408 KiB After Width: | Height: | Size: 388 KiB |
+2095
-4469
File diff suppressed because it is too large
Load Diff
|
Before Width: | Height: | Size: 514 KiB After Width: | Height: | Size: 438 KiB |
+9
-6
@@ -386,18 +386,21 @@ def main() -> None:
|
||||
# Drop the " (rated)" readout tag from the on-map labels (the cache/CI-table keep it) -- the map is
|
||||
# crowded and every model here is rated, so the tag adds nothing.
|
||||
plot_models = {k.replace(" (rated)", ""): v for k, v in models.items()}
|
||||
# Too many model names to label them all. Plot every star (colour = family) but LABEL only the
|
||||
# latest model per family (highest version number), and drop the redundant "claude-" so the flagship
|
||||
# reads "opus-4.8". Colour + legend carry the unlabelled siblings.
|
||||
# Too many model names to label them all. Plot every star (colour = family) but label only the
|
||||
# latest catalogued release in each family. Numeric version strings do not establish release order.
|
||||
fams: dict[str, list[str]] = {}
|
||||
for k in plot_models:
|
||||
family = maps.model_family(k)
|
||||
if family is None:
|
||||
raise ValueError(f"model has no explicit family: {k}")
|
||||
fams.setdefault(family, []).append(k)
|
||||
def _ver(k: str) -> list[float]:
|
||||
return [float(n) for n in re.findall(r"\d+(?:\.\d+)?", k)]
|
||||
model_labels = {max(ks, key=_ver): max(ks, key=_ver).replace("claude-", "") for ks in fams.values()}
|
||||
metadata = json.loads(Path("docs/img/wvs/wvs_model_metadata.json").read_text())["models"]
|
||||
model_labels: dict[str, str] = {}
|
||||
for family, names in fams.items():
|
||||
catalogued = [name for name in names if name in metadata]
|
||||
if catalogued:
|
||||
latest = max(catalogued, key=lambda name: metadata[name]["created"])
|
||||
model_labels[latest] = latest.replace("claude-", "")
|
||||
# Poles in NATURAL data order (x_neg, x_pos, y_neg, y_pos): raw X is high on Self-expression, raw Y
|
||||
# high on Secular-Rational. plot_value_map's orient_geographic then flips X so the cultural West
|
||||
# lands in the west (Self-expression left) and confirms African-Islamic sits south -- the same
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -2,6 +2,10 @@
|
||||
|
||||
Checked from the saved OpenRouter catalog on 2026-09-16. Protocol IDs hash all rendered WVS prompts and settings. `response_format` is only requested where the catalog advertises it. A mandatory model uses its least listed effort. Entries with mandatory reasoning but no supported-effort list are excluded rather than guessing a setting. These are candidate runs, not completed points.
|
||||
|
||||
## Actual mandatory-reasoning batch
|
||||
|
||||
The executed mandatory batch used one shared setting, `reasoning={"effort":"low"}`, with strict JSON schema and concurrency 1 for `anthropic/claude-fable-5.1`, `openai/gpt-6-astra`, `meta/muse-spark-1.3`, `z-ai/glm-5.3`, `z-ai/glm-5.3-flash`, `google/gemini-3.7-flash`, and `x-ai/grok-4.5`. Therefore the candidate hash for Muse below, which uses the catalog minimum `minimal`, is not the executed protocol. The append-only request ledger is the source of truth for actual run ID, protocol hash, raw request settings, usage and errors. -- PI[gpt-5.6-terra]
|
||||
|
||||
| exact ID | catalog name | created UTC | input USD/M | output USD/M | reasoning | schema | protocol ID |
|
||||
|---|---|---:|---:|---:|---|---|---|
|
||||
| `anthropic/claude-fable-5.1` | Anthropic: Claude Fable 5.1 | 2026-09-01 | 10 | 50 | mandatory low | True | `4230ccaa1e160c793a906aef6b786d735d9fdbf71c3333691b0eba17dcf33928` |
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
# Fresh-eyes review: WVS static map
|
||||
|
||||
Artifact reviewed: `docs/img/wvs/wvs_map_iw.png`.
|
||||
|
||||
Reviewer: `reviewer-anthropic`, read-only image review, 2026-09-16. This is an observation of the rendered image, not an audit of data or code.
|
||||
|
||||
> Overall readability: moderate. The axes, quadrant labels, title, and cultural-zone hulls are clear. The upper-left is badly cluttered.
|
||||
|
||||
> Orange overload / color collision. The many orange stars share the exact hue of the Latin America hull.
|
||||
|
||||
> Label-to-marker ambiguity. Model labels float without leader lines and several are far from any same-colored star.
|
||||
|
||||
> `deepseek-v4.1-flash` label overlaps `Great Britain` label. `mistral-large-2512` and `llama-4-maverick` labels stack tightly with `Sweden`.
|
||||
|
||||
The latest rendered image now says `64 models`, so the reviewer's earlier caption observation (`17 models`) has been corrected. The color collision, crowded upper-left, and label ambiguity remain visible in the latest PNG. The static geometry is interpretable but this does not pass the plan's readability discriminator for a published interactive page. No Pages deployment was attempted. -- PI[gpt-5.6-terra]
|
||||
Reference in New Issue
Block a user