From 94dbb117975422dcb2e5f9a17af843f7a69d0fd7 Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Thu, 9 Jul 2026 09:51:57 +0800 Subject: [PATCH] README: fix alien claims to match the plots (openness+agreeableness low, not conscientiousness; sweep spans most of human range, not more than any country pair) Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com> --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index ac5d00f..54f35f7 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,13 @@ Fast value evals for local language models. It asks standard human survey questions and moral vignettes, reads the model's answer-token probabilities, and compares the profile to human norms. Built for steering work: the readout is sensitive enough that a small intervention shows up as a probability shift. -The instruments are standard human ones, bundled with human reference data: World Values Survey items (via [GlobalOpinionQA](https://huggingface.co/datasets/Anthropic/llm_global_opinions)), [moral-foundation vignettes](https://scottaclifford.com/wp-content/uploads/2015/01/CICSA_MoralVignettes_BRM_ND.pdf) (Clifford et al. 2015), MFQ-2, Big Five, 16PF, and Humor Styles. An example item, from the World Values Survey: +The instruments are standard human ones, bundled with human reference data: World Values Survey items (via [GlobalOpinionQA](https://huggingface.co/datasets/Anthropic/llm_global_opinions)), [moral-foundation vignettes](https://scottaclifford.com/wp-content/uploads/2015/01/CICSA_MoralVignettes_BRM_ND.pdf) (Clifford et al. 2015, the repo's namesake), MFQ-2, Big Five, 16PF, and Humor Styles. An example item, from the World Values Survey: > Generally speaking, would you say that most people can be trusted or that you need to be very careful in dealing with people? ## Are models moral aliens? -Before any steering the base model is already a psychological alien on some axes: on the culture maps it sits away from most humans, most sharply on Big5 openness (low) and conscientiousness (high), and on humor style (high aggressive, low affiliative). And steering is strong relative to human variation: on the headline axes a single sweep moves the profile further than any two countries differ. +Two things jump out of the plots below. First, before any steering the models are already psychological outliers. All seventeen frontier models sit outside the cultural mean of nearly every country on the World Values Survey map: more secular and more self-expression-leaning than almost any human society, an ultra Silicon Valley cultural point. And the open model we probe in depth, Qwen3-4B, scores below every surveyed country on Big Five openness and agreeableness, and reports more aggressive and less affiliative humor than every country except Malaysia. Second, steering is strong relative to human variation: on MFQ-2 a single sweep walks the model across most of the human range. ### The whole field, on the world's value map @@ -16,7 +16,7 @@ The clearest single view comes from the World Values Survey, the standard cultur ![WVS culture map: 17 frontier models among about 90 human societies](docs/img/wvs/wvs_map_iw.png) -Every model sits in the top-left: more secular and more self-expressive than almost any country on earth, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies. This is a different reading from the rest of the page. These frontier models are closed APIs with no answer probabilities to read, so each is scored by rated sampling (rate every option one to five, twelve times, with the option order shuffled), and the human positions are approximated from the GlobalOpinionQA question set. The steering plots below instead follow one open model we can push, Qwen3-4B. +Every model sits in the top-left: more secular and more self-expressive than almost any country on earth, deep in the rich-world corner and often past its edge, and none of them sits near the African or Muslim societies. This map is measured differently from everything else on the page. These frontier models are closed APIs with no answer probabilities to read, so each is scored by rated sampling (rate every option one to five, twelve times, with the option order shuffled), and the human positions are approximated from the GlobalOpinionQA question set. The steering plots below instead follow one open model we can push, Qwen3-4B. What happens when we steer them? Below we steer models with `authority-respecting` versus `authority-disregarding` personas.