wassname/moral-maps
1
0
Fork 0
mirror of https://github.com/wassname/moral-maps.git synced 2026-09-21 13:10:52 +08:00
Code Issues Packages Projects Releases Wiki Activity

Regenerate WVS map from complete panels

Browse Source 
Co-Authored-By: PI[gpt-5.6-terra] <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-09-17 04:28:14 +08:00
co-authored by PI[gpt-5.6-terra]
parent f2d6556af1
commit 8db3d3daa9
16 changed files with 33524 additions and 4787 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/RESEARCH_JOURNAL.md
+27
View File
@@ -911,3 +911,30 @@ prove total spend remains below the authorized cap, so durable records must reta
and request phase before the next paid call. -- PI[gpt-5.6-terra]
The next result will replace these bounds with reconciled provider-reported usage.
## 2026-09-16 -- WVS measured OpenRouter usage
This entry records what the durable WVS request ledger reports after the API panels.
The source ledger is `slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl`. It contains 48 complete
panels and eleven incomplete attempts. Summing the provider `usage.cost` field across every completed request
phase, including excluded attempts, gives USD 3.59086067. This is provider-reported per-request cost, not a
completion-token estimate. The first Qwen diagnostic cost USD 0.04310443 and produced 129 valid samples after
142 rescues. The corrected Qwen 3.7 Flash panel used `reasoning.enabled=false` plus strict JSON schema, cost
USD 0.00110376, and has 144 distinct item/sample keys, 144 valid parses, zero rescues, and zero request failures.
Source: `qwen3.7-flash_schema_audit.md` and the raw ledger in the same directory.
The provider usage records expose prompt tokens, completion tokens, total tokens, cost and generation IDs, but
some omit separate reasoning and cache fields. Those fields remain unknown rather than being counted as zero.
Each completed cache entry records its exact model, prompt-and-settings protocol hash, contributing run ID and
coordinate result. The execution matrix in `execution_matrix.md` separates optional reasoning disabled, mandatory
reasoning at a listed low effort, no advertised reasoning control, and catalog entries whose mandatory effort is
unknown. It is a catalog snapshot, not a billing statement.
My read: the strict schema plus disabled optional reasoning is probably the protocol change that made the Qwen
panel complete, because the prior same-model attempts stored partial JSON objects and spent their output allowance
on reasoning. Several provider/network or deterministic-format failures remain excluded from plotted coordinates.
The saved ledger permits a later exact-protocol completion calculation without reinterpreting an incomplete panel as
a valid point. -- PI[gpt-5.6-terra]
The durable records make the measured spend and exclusions available for the next decision.
docs/img/wvs/wvs_map_iw.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 350 KiB

After

Width:  |  Height:  |  Size: 408 KiB

docs/img/wvs/wvs_map_iw.svg
+6985 -4660
View File
File diff suppressed because it is too large Load Diff
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 434 KiB

After

Width:  |  Height:  |  Size: 514 KiB

docs/img/wvs/wvs_model_ci.md
+66 -19
View File
@@ -1,19 +1,66 @@
| model | x self-expr | y secular | x 95%CI | y 95%CI |
|:---------------------------|--------------:|------------:|----------:|----------:|
| qwen3.7-max (rated) | +0.37 | +0.69 | +0.16 | +0.17 |
| grok-4.20 (rated) | +0.59 | +0.62 | +0.11 | +0.17 |
| gemma-4-31b-it (rated) | +0.36 | +0.66 | +0.16 | +0.13 |
| gemini-2.5-pro (rated) | +0.47 | +0.70 | +0.13 | +0.14 |
| grok-4.3 (rated) | +0.44 | +0.73 | +0.14 | +0.13 |
| deepseek-v4-flash (rated) | +0.57 | +0.64 | +0.11 | +0.16 |
| gpt-5.4 (rated) | +0.45 | +0.68 | +0.13 | +0.14 |
| mistral-large-2512 (rated) | +0.65 | +0.64 | +0.08 | +0.18 |
| gemma-3-27b-it (rated) | +0.60 | +0.60 | +0.09 | +0.17 |
| llama-4-maverick (rated) | +0.64 | +0.64 | +0.08 | +0.18 |
| gpt-5.3-chat (rated) | +0.50 | +0.67 | +0.09 | +0.16 |
| deepseek-v4-pro (rated) | +0.55 | +0.73 | +0.15 | +0.10 |
| llama-4-scout (rated) | +0.62 | +0.53 | +0.05 | +0.20 |
| gpt-5.5 (rated) | +0.42 | +0.76 | +0.17 | +0.07 |
| claude-opus-4.7 (rated) | +0.59 | +0.60 | +0.07 | +0.09 |
| claude-opus-4.8 (rated) | +0.61 | +0.58 | +0.07 | +0.07 |
| claude-opus-4.6 (rated) | +0.63 | +0.63 | +0.04 | +0.05 |
| model | x self-expr | y secular | x 95%CI | y 95%CI |
|:-------------------------------------|--------------:|------------:|----------:|----------:|
| qwen3.6-27b (rated) | +0.46 | +0.68 | +0.18 | +0.14 |
| qwen3.5-35b-a3b (rated) | +0.54 | +0.55 | +0.14 | +0.18 |
| qwen3.6-max-preview (rated) | +0.48 | +0.69 | +0.16 | +0.14 |
| muse-spark-1.3 (rated) | +0.43 | +0.73 | +0.18 | +0.12 |
| qwen3.8-27b (rated) | +0.43 | +0.69 | +0.16 | +0.14 |
| qwen3.8-flash (rated) | +0.42 | +0.63 | +0.14 | +0.15 |
| gemma-4-31b-it (rated) | +0.36 | +0.66 | +0.16 | +0.13 |
| qwen3.5-122b-a10b (rated) | +0.48 | +0.62 | +0.15 | +0.14 |
| grok-4.20 (rated) | +0.59 | +0.62 | +0.11 | +0.17 |
| qwen3.5-plus-20260420 (rated) | +0.59 | +0.65 | +0.12 | +0.16 |
| gemini-2.5-pro (rated) | +0.47 | +0.70 | +0.13 | +0.14 |
| grok-4.3 (rated) | +0.44 | +0.73 | +0.14 | +0.13 |
| deepseek-v4-flash (rated) | +0.57 | +0.64 | +0.11 | +0.16 |
| gpt-5.4 (rated) | +0.45 | +0.68 | +0.13 | +0.14 |
| deepseek-v4.1-flash (rated) | +0.54 | +0.60 | +0.10 | +0.16 |
| qwen3-coder-30b-a3b-instruct (rated) | +0.50 | +0.58 | +0.09 | +0.17 |
| qwen3-30b-a3b (rated) | +0.67 | +0.61 | +0.07 | +0.19 |
| gpt-6-astra (rated) | +0.46 | +0.68 | +0.15 | +0.11 |
| mistral-large-2512 (rated) | +0.65 | +0.64 | +0.08 | +0.18 |
| gemma-3-27b-it (rated) | +0.60 | +0.60 | +0.09 | +0.17 |
| llama-4-maverick (rated) | +0.64 | +0.64 | +0.08 | +0.18 |
| qwen3-coder (rated) | +0.60 | +0.62 | +0.08 | +0.18 |
| qwen3.7-flash (rated) | +0.65 | +0.59 | +0.10 | +0.16 |
| qwen3-vl-235b-a22b-instruct (rated) | +0.60 | +0.55 | +0.05 | +0.21 |
| qwen-plus (rated) | +0.60 | +0.63 | +0.07 | +0.18 |
| gpt-5.3-chat (rated) | +0.50 | +0.67 | +0.09 | +0.16 |
| deepseek-v4-pro (rated) | +0.55 | +0.73 | +0.15 | +0.10 |
| llama-4-scout (rated) | +0.62 | +0.53 | +0.05 | +0.20 |
| qwen3-vl-30b-a3b-instruct (rated) | +0.53 | +0.53 | +0.06 | +0.19 |
| qwen3.5-397b-a17b (rated) | +0.54 | +0.65 | +0.08 | +0.17 |
| qwen3.5-plus-02-15 (rated) | +0.54 | +0.66 | +0.08 | +0.17 |
| glm-5.3 (rated) | +0.55 | +0.64 | +0.12 | +0.13 |
| qwen3.7-plus (rated) | +0.51 | +0.67 | +0.10 | +0.15 |
| qwen3-30b-a3b-instruct-2507 (rated) | +0.65 | +0.54 | +0.05 | +0.20 |
| gpt-5.5 (rated) | +0.42 | +0.76 | +0.17 | +0.07 |
| qwen3.6-35b-a3b (rated) | +0.61 | +0.59 | +0.10 | +0.14 |
| qwen3.6-flash (rated) | +0.65 | +0.58 | +0.09 | +0.15 |
| qwen3.6-plus (rated) | +0.52 | +0.67 | +0.08 | +0.15 |
| qwen3-235b-a22b (rated) | +0.64 | +0.56 | +0.07 | +0.17 |
| qwen-2.5-72b-instruct (rated) | +0.60 | +0.60 | +0.06 | +0.17 |
| qwen-plus-2025-07-28 (rated) | +0.64 | +0.51 | +0.07 | +0.16 |
| qwen3.7-max (rated) | +0.50 | +0.65 | +0.11 | +0.12 |
| qwen3-235b-a22b-2507 (rated) | +0.64 | +0.57 | +0.07 | +0.16 |
| qwen3.5-9b (rated) | +0.50 | +0.59 | +0.09 | +0.14 |
| qwen2.5-vl-72b-instruct (rated) | +0.58 | +0.60 | +0.07 | +0.15 |
| qwen3-vl-32b-instruct (rated) | +0.60 | +0.53 | +0.02 | +0.20 |
| gpt-5.6-sol (rated) | +0.53 | +0.67 | +0.09 | +0.13 |
| qwen3-max-thinking (rated) | +0.58 | +0.70 | +0.07 | +0.14 |
| qwen3-coder-plus (rated) | +0.62 | +0.60 | +0.07 | +0.14 |
| inkling (rated) | +0.56 | +0.69 | +0.10 | +0.11 |
| qwen3-vl-8b-instruct (rated) | +0.61 | +0.60 | +0.06 | +0.15 |
| qwen-2.5-7b-instruct (rated) | +0.55 | +0.59 | +0.05 | +0.16 |
| kimi-k3 (rated) | +0.62 | +0.67 | +0.07 | +0.14 |
| qwen3.5-27b (rated) | +0.60 | +0.65 | +0.05 | +0.16 |
| qwen3-32b (rated) | +0.61 | +0.54 | +0.06 | +0.14 |
| qwen3-14b (rated) | +0.61 | +0.58 | +0.03 | +0.16 |
| qwen3-coder-next (rated) | +0.64 | +0.58 | +0.05 | +0.13 |
| qwen3-next-80b-a3b-instruct (rated) | +0.67 | +0.70 | +0.05 | +0.12 |
| claude-opus-4.7 (rated) | +0.59 | +0.60 | +0.07 | +0.09 |
| qwen3-8b (rated) | +0.62 | +0.54 | +0.03 | +0.12 |
| claude-opus-4.8 (rated) | +0.61 | +0.58 | +0.07 | +0.07 |
| gemini-3.7-flash (rated) | +0.50 | +0.63 | +0.03 | +0.10 |
| claude-fable-5.1 (rated) | +0.58 | +0.61 | +0.07 | +0.05 |
| claude-opus-4.6 (rated) | +0.63 | +0.63 | +0.04 | +0.05 |
docs/img/wvs/wvs_model_outlier_sd.md
+322 -87
View File
@@ -1,87 +1,322 @@
| model | zone | n countries | z self-expr | z secular | Mahalanobis |
|:-------------------|:----------------|--------------:|--------------:|------------:|--------------:|
| gpt-5.5 | African-Islamic | 19 | +2.36 | +6.12 | +6.51 |
| gpt-5.5 | East Asia | 9 | +1.31 | +3.54 | +3.72 |
| gpt-5.5 | Latin America | 14 | +2.35 | +6.45 | +6.51 |
| gpt-5.5 | Orthodox | 14 | +2.94 | +8.06 | +8.52 |
| gpt-5.5 | West | 29 | -0.33 | +2.93 | +4.71 |
| deepseek-v4-pro | African-Islamic | 19 | +3.70 | +5.75 | +5.75 |
| deepseek-v4-pro | East Asia | 9 | +3.10 | +3.14 | +3.46 |
| deepseek-v4-pro | Latin America | 14 | +4.25 | +6.05 | +6.74 |
| deepseek-v4-pro | Orthodox | 14 | +5.25 | +7.48 | +9.04 |
| deepseek-v4-pro | West | 29 | +0.54 | +2.61 | +3.32 |
| grok-4.3 | African-Islamic | 19 | +2.57 | +5.75 | +5.98 |
| grok-4.3 | East Asia | 9 | +1.58 | +3.14 | +3.17 |
| grok-4.3 | Latin America | 14 | +2.64 | +6.05 | +6.19 |
| grok-4.3 | Orthodox | 14 | +3.29 | +7.48 | +8.11 |
| grok-4.3 | West | 29 | -0.19 | +2.61 | +4.08 |
| gemini-2.5-pro | African-Islamic | 19 | +2.88 | +5.39 | +5.46 |
| gemini-2.5-pro | East Asia | 9 | +2.00 | +2.73 | +2.76 |
| gemini-2.5-pro | Latin America | 14 | +3.08 | +5.66 | +5.95 |
| gemini-2.5-pro | Orthodox | 14 | +3.83 | +6.89 | +7.81 |
| gemini-2.5-pro | West | 29 | +0.01 | +2.29 | +3.38 |
| qwen3.7-max | African-Islamic | 19 | +1.85 | +5.27 | +5.69 |
| qwen3.7-max | East Asia | 9 | +0.62 | +2.59 | +2.89 |
| qwen3.7-max | Latin America | 14 | +1.62 | +5.52 | +5.54 |
| qwen3.7-max | Orthodox | 14 | +2.05 | +6.70 | +6.96 |
| qwen3.7-max | West | 29 | -0.66 | +2.18 | +4.01 |
| gpt-5.4 | African-Islamic | 19 | +2.67 | +5.14 | +5.23 |
| gpt-5.4 | East Asia | 9 | +1.72 | +2.46 | +2.47 |
| gpt-5.4 | Latin America | 14 | +2.79 | +5.39 | +5.63 |
| gpt-5.4 | Orthodox | 14 | +3.47 | +6.50 | +7.30 |
| gpt-5.4 | West | 29 | -0.13 | +2.07 | +3.21 |
| gpt-5.3-chat | African-Islamic | 19 | +3.19 | +5.02 | +5.03 |
| gpt-5.3-chat | East Asia | 9 | +2.41 | +2.32 | +2.63 |
| gpt-5.3-chat | Latin America | 14 | +3.52 | +5.26 | +5.78 |
| gpt-5.3-chat | Orthodox | 14 | +4.36 | +6.31 | +7.59 |
| gpt-5.3-chat | West | 29 | +0.21 | +1.96 | +2.69 |
| gemma-4-31b-it | African-Islamic | 19 | +1.74 | +4.90 | +5.29 |
| gemma-4-31b-it | East Asia | 9 | +0.48 | +2.19 | +2.46 |
| gemma-4-31b-it | Latin America | 14 | +1.47 | +5.13 | +5.14 |
| gemma-4-31b-it | Orthodox | 14 | +1.87 | +6.11 | +6.36 |
| gemma-4-31b-it | West | 29 | -0.73 | +1.86 | +3.62 |
| deepseek-v4-flash | African-Islamic | 19 | +3.91 | +4.66 | +4.79 |
| deepseek-v4-flash | East Asia | 9 | +3.38 | +1.91 | +3.39 |
| deepseek-v4-flash | Latin America | 14 | +4.54 | +4.87 | +6.02 |
| deepseek-v4-flash | Orthodox | 14 | +5.60 | +5.72 | +7.92 |
| deepseek-v4-flash | West | 29 | +0.68 | +1.64 | +1.83 |
| llama-4-maverick | African-Islamic | 19 | +4.63 | +4.66 | +5.10 |
| llama-4-maverick | East Asia | 9 | +4.34 | +1.91 | +4.46 |
| llama-4-maverick | Latin America | 14 | +5.56 | +4.87 | +6.69 |
| llama-4-maverick | Orthodox | 14 | +6.84 | +5.72 | +8.82 |
| llama-4-maverick | West | 29 | +1.14 | +1.64 | +1.65 |
| mistral-large-2512 | African-Islamic | 19 | +4.73 | +4.66 | +5.15 |
| mistral-large-2512 | East Asia | 9 | +4.48 | +1.91 | +4.62 |
| mistral-large-2512 | Latin America | 14 | +5.71 | +4.87 | +6.79 |
| mistral-large-2512 | Orthodox | 14 | +7.02 | +5.72 | +8.96 |
| mistral-large-2512 | West | 29 | +1.21 | +1.64 | +1.64 |
| claude-opus-4.6 | African-Islamic | 19 | +4.53 | +4.54 | +4.97 |
| claude-opus-4.6 | East Asia | 9 | +4.21 | +1.78 | +4.34 |
| claude-opus-4.6 | Latin America | 14 | +5.42 | +4.73 | +6.51 |
| claude-opus-4.6 | Orthodox | 14 | +6.67 | +5.53 | +8.57 |
| claude-opus-4.6 | West | 29 | +1.08 | +1.54 | +1.54 |
| grok-4.20 | African-Islamic | 19 | +4.11 | +4.42 | +4.70 |
| grok-4.20 | East Asia | 9 | +3.65 | +1.64 | +3.74 |
| grok-4.20 | Latin America | 14 | +4.83 | +4.60 | +6.03 |
| grok-4.20 | Orthodox | 14 | +5.96 | +5.33 | +7.91 |
| grok-4.20 | West | 29 | +0.81 | +1.43 | +1.47 |
| claude-opus-4.7 | African-Islamic | 19 | +4.11 | +4.17 | +4.55 |
| claude-opus-4.7 | East Asia | 9 | +3.65 | +1.37 | +3.83 |
| claude-opus-4.7 | Latin America | 14 | +4.83 | +4.34 | +5.87 |
| claude-opus-4.7 | Orthodox | 14 | +5.96 | +4.94 | +7.66 |
| claude-opus-4.7 | West | 29 | +0.81 | +1.21 | +1.22 |
| gemma-3-27b-it | African-Islamic | 19 | +4.22 | +4.17 | +4.60 |
| gemma-3-27b-it | East Asia | 9 | +3.79 | +1.37 | +4.00 |
| gemma-3-27b-it | Latin America | 14 | +4.98 | +4.34 | +5.97 |
| gemma-3-27b-it | Orthodox | 14 | +6.13 | +4.94 | +7.79 |
| gemma-3-27b-it | West | 29 | +0.88 | +1.21 | +1.21 |
| claude-opus-4.8 | African-Islamic | 19 | +4.32 | +3.93 | +4.55 |
| claude-opus-4.8 | East Asia | 9 | +3.93 | +1.10 | +4.29 |
| claude-opus-4.8 | Latin America | 14 | +5.13 | +4.08 | +5.94 |
| claude-opus-4.8 | Orthodox | 14 | +6.31 | +4.55 | +7.70 |
| claude-opus-4.8 | West | 29 | +0.94 | +1.00 | +1.04 |
| llama-4-scout | African-Islamic | 19 | +4.42 | +3.32 | +4.46 |
| llama-4-scout | East Asia | 9 | +4.07 | +0.42 | +4.88 |
| llama-4-scout | Latin America | 14 | +5.27 | +3.42 | +5.75 |
| llama-4-scout | Orthodox | 14 | +6.49 | +3.58 | +7.34 |
| llama-4-scout | West | 29 | +1.01 | +0.46 | +1.09 |
| model | zone | n countries | z self-expr | z secular | Mahalanobis |
|:-----------------------------|:----------------|--------------:|--------------:|------------:|--------------:|
| gpt-5.5 | African-Islamic | 19 | +2.36 | +6.12 | +6.51 |
| gpt-5.5 | East Asia | 9 | +1.31 | +3.54 | +3.72 |
| gpt-5.5 | Latin America | 14 | +2.35 | +6.45 | +6.51 |
| gpt-5.5 | Orthodox | 14 | +2.94 | +8.06 | +8.52 |
| gpt-5.5 | West | 29 | -0.33 | +2.93 | +4.71 |
| deepseek-v4-pro | African-Islamic | 19 | +3.70 | +5.75 | +5.75 |
| deepseek-v4-pro | East Asia | 9 | +3.10 | +3.14 | +3.46 |
| deepseek-v4-pro | Latin America | 14 | +4.25 | +6.05 | +6.74 |
| deepseek-v4-pro | Orthodox | 14 | +5.25 | +7.48 | +9.04 |
| deepseek-v4-pro | West | 29 | +0.54 | +2.61 | +3.32 |
| grok-4.3 | African-Islamic | 19 | +2.57 | +5.75 | +5.98 |
| grok-4.3 | East Asia | 9 | +1.58 | +3.14 | +3.17 |
| grok-4.3 | Latin America | 14 | +2.64 | +6.05 | +6.19 |
| grok-4.3 | Orthodox | 14 | +3.29 | +7.48 | +8.11 |
| grok-4.3 | West | 29 | -0.19 | +2.61 | +4.08 |
| muse-spark-1.3 | African-Islamic | 19 | +2.47 | +5.75 | +6.02 |
| muse-spark-1.3 | East Asia | 9 | +1.45 | +3.14 | +3.20 |
| muse-spark-1.3 | Latin America | 14 | +2.50 | +6.05 | +6.16 |
| muse-spark-1.3 | Orthodox | 14 | +3.12 | +7.48 | +8.04 |
| muse-spark-1.3 | West | 29 | -0.26 | +2.61 | +4.16 |
| gemini-2.5-pro | African-Islamic | 19 | +2.88 | +5.39 | +5.46 |
| gemini-2.5-pro | East Asia | 9 | +2.00 | +2.73 | +2.76 |
| gemini-2.5-pro | Latin America | 14 | +3.08 | +5.66 | +5.95 |
| gemini-2.5-pro | Orthodox | 14 | +3.83 | +6.89 | +7.81 |
| gemini-2.5-pro | West | 29 | +0.01 | +2.29 | +3.38 |
| qwen3-max-thinking | African-Islamic | 19 | +4.01 | +5.39 | +5.42 |
| qwen3-max-thinking | East Asia | 9 | +3.52 | +2.73 | +3.58 |
| qwen3-max-thinking | Latin America | 14 | +4.69 | +5.66 | +6.65 |
| qwen3-max-thinking | Orthodox | 14 | +5.78 | +6.89 | +8.90 |
| qwen3-max-thinking | West | 29 | +0.74 | +2.29 | +2.68 |
| qwen3-next-80b-a3b-instruct | African-Islamic | 19 | +4.94 | +5.39 | +5.69 |
| qwen3-next-80b-a3b-instruct | East Asia | 9 | +4.76 | +2.73 | +4.77 |
| qwen3-next-80b-a3b-instruct | Latin America | 14 | +6.00 | +5.66 | +7.45 |
| qwen3-next-80b-a3b-instruct | Orthodox | 14 | +7.38 | +6.89 | +9.98 |
| qwen3-next-80b-a3b-instruct | West | 29 | +1.34 | +2.29 | +2.34 |
| inkling | African-Islamic | 19 | +3.81 | +5.27 | +5.28 |
| inkling | East Asia | 9 | +3.24 | +2.59 | +3.32 |
| inkling | Latin America | 14 | +4.40 | +5.52 | +6.40 |
| inkling | Orthodox | 14 | +5.42 | +6.70 | +8.53 |
| inkling | West | 29 | +0.61 | +2.18 | +2.64 |
| qwen3.6-max-preview | African-Islamic | 19 | +2.98 | +5.27 | +5.31 |
| qwen3.6-max-preview | East Asia | 9 | +2.14 | +2.59 | +2.68 |
| qwen3.6-max-preview | Latin America | 14 | +3.23 | +5.52 | +5.89 |
| qwen3.6-max-preview | Orthodox | 14 | +4.00 | +6.70 | +7.73 |
| qwen3.6-max-preview | West | 29 | +0.07 | +2.18 | +3.15 |
| qwen3.8-27b | African-Islamic | 19 | +2.47 | +5.27 | +5.43 |
| qwen3.8-27b | East Asia | 9 | +1.45 | +2.59 | +2.60 |
| qwen3.8-27b | Latin America | 14 | +2.50 | +5.52 | +5.67 |
| qwen3.8-27b | Orthodox | 14 | +3.12 | +6.70 | +7.33 |
| qwen3.8-27b | West | 29 | -0.26 | +2.18 | +3.53 |
| gpt-5.4 | African-Islamic | 19 | +2.67 | +5.14 | +5.23 |
| gpt-5.4 | East Asia | 9 | +1.72 | +2.46 | +2.47 |
| gpt-5.4 | Latin America | 14 | +2.79 | +5.39 | +5.63 |
| gpt-5.4 | Orthodox | 14 | +3.47 | +6.50 | +7.30 |
| gpt-5.4 | West | 29 | -0.13 | +2.07 | +3.21 |
| gpt-6-astra | African-Islamic | 19 | +2.77 | +5.14 | +5.21 |
| gpt-6-astra | East Asia | 9 | +1.86 | +2.46 | +2.49 |
| gpt-6-astra | Latin America | 14 | +2.93 | +5.39 | +5.67 |
| gpt-6-astra | Orthodox | 14 | +3.65 | +6.50 | +7.39 |
| gpt-6-astra | West | 29 | -0.06 | +2.07 | +3.14 |
| qwen3.6-27b | African-Islamic | 19 | +2.77 | +5.14 | +5.21 |
| qwen3.6-27b | East Asia | 9 | +1.86 | +2.46 | +2.49 |
| qwen3.6-27b | Latin America | 14 | +2.93 | +5.39 | +5.67 |
| qwen3.6-27b | Orthodox | 14 | +3.65 | +6.50 | +7.39 |
| qwen3.6-27b | West | 29 | -0.06 | +2.07 | +3.14 |
| gpt-5.3-chat | African-Islamic | 19 | +3.19 | +5.02 | +5.03 |
| gpt-5.3-chat | East Asia | 9 | +2.41 | +2.32 | +2.63 |
| gpt-5.3-chat | Latin America | 14 | +3.52 | +5.26 | +5.78 |
| gpt-5.3-chat | Orthodox | 14 | +4.36 | +6.31 | +7.59 |
| gpt-5.3-chat | West | 29 | +0.21 | +1.96 | +2.69 |
| gpt-5.6-sol | African-Islamic | 19 | +3.50 | +5.02 | +5.03 |
| gpt-5.6-sol | East Asia | 9 | +2.83 | +2.32 | +2.92 |
| gpt-5.6-sol | Latin America | 14 | +3.96 | +5.26 | +5.98 |
| gpt-5.6-sol | Orthodox | 14 | +4.89 | +6.31 | +7.90 |
| gpt-5.6-sol | West | 29 | +0.41 | +1.96 | +2.50 |
| kimi-k3 | African-Islamic | 19 | +4.42 | +5.02 | +5.23 |
| kimi-k3 | East Asia | 9 | +4.07 | +2.32 | +4.08 |
| kimi-k3 | Latin America | 14 | +5.27 | +5.26 | +6.73 |
| kimi-k3 | Orthodox | 14 | +6.49 | +6.31 | +8.95 |
| kimi-k3 | West | 29 | +1.01 | +1.96 | +2.07 |
| qwen3.6-plus | African-Islamic | 19 | +3.39 | +5.02 | +5.02 |
| qwen3.6-plus | East Asia | 9 | +2.69 | +2.32 | +2.81 |
| qwen3.6-plus | Latin America | 14 | +3.81 | +5.26 | +5.91 |
| qwen3.6-plus | Orthodox | 14 | +4.71 | +6.31 | +7.79 |
| qwen3.6-plus | West | 29 | +0.34 | +1.96 | +2.56 |
| qwen3.7-plus | African-Islamic | 19 | +3.29 | +5.02 | +5.02 |
| qwen3.7-plus | East Asia | 9 | +2.55 | +2.32 | +2.72 |
| qwen3.7-plus | Latin America | 14 | +3.66 | +5.26 | +5.84 |
| qwen3.7-plus | Orthodox | 14 | +4.54 | +6.31 | +7.69 |
| qwen3.7-plus | West | 29 | +0.27 | +1.96 | +2.63 |
| gemma-4-31b-it | African-Islamic | 19 | +1.74 | +4.90 | +5.29 |
| gemma-4-31b-it | East Asia | 9 | +0.48 | +2.19 | +2.46 |
| gemma-4-31b-it | Latin America | 14 | +1.47 | +5.13 | +5.14 |
| gemma-4-31b-it | Orthodox | 14 | +1.87 | +6.11 | +6.36 |
| gemma-4-31b-it | West | 29 | -0.73 | +1.86 | +3.62 |
| qwen3.5-plus-02-15 | African-Islamic | 19 | +3.60 | +4.90 | +4.93 |
| qwen3.5-plus-02-15 | East Asia | 9 | +2.96 | +2.19 | +3.00 |
| qwen3.5-plus-02-15 | Latin America | 14 | +4.10 | +5.13 | +5.96 |
| qwen3.5-plus-02-15 | Orthodox | 14 | +5.07 | +6.11 | +7.85 |
| qwen3.5-plus-02-15 | West | 29 | +0.47 | +1.86 | +2.28 |
| qwen3.5-27b | African-Islamic | 19 | +4.22 | +4.78 | +4.98 |
| qwen3.5-27b | East Asia | 9 | +3.79 | +2.05 | +3.81 |
| qwen3.5-27b | Latin America | 14 | +4.98 | +5.00 | +6.37 |
| qwen3.5-27b | Orthodox | 14 | +6.13 | +5.92 | +8.43 |
| qwen3.5-27b | West | 29 | +0.88 | +1.75 | +1.86 |
| qwen3.5-397b-a17b | African-Islamic | 19 | +3.60 | +4.78 | +4.82 |
| qwen3.5-397b-a17b | East Asia | 9 | +2.96 | +2.05 | +2.98 |
| qwen3.5-397b-a17b | Latin America | 14 | +4.10 | +5.00 | +5.86 |
| qwen3.5-397b-a17b | Orthodox | 14 | +5.07 | +5.92 | +7.71 |
| qwen3.5-397b-a17b | West | 29 | +0.47 | +1.75 | +2.13 |
| qwen3.5-plus-20260420 | African-Islamic | 19 | +4.11 | +4.78 | +4.95 |
| qwen3.5-plus-20260420 | East Asia | 9 | +3.65 | +2.05 | +3.66 |
| qwen3.5-plus-20260420 | Latin America | 14 | +4.83 | +5.00 | +6.28 |
| qwen3.5-plus-20260420 | Orthodox | 14 | +5.96 | +5.92 | +8.30 |
| qwen3.5-plus-20260420 | West | 29 | +0.81 | +1.75 | +1.89 |
| qwen3.7-max | African-Islamic | 19 | +3.19 | +4.78 | +4.78 |
| qwen3.7-max | East Asia | 9 | +2.41 | +2.05 | +2.51 |
| qwen3.7-max | Latin America | 14 | +3.52 | +5.00 | +5.57 |
| qwen3.7-max | Orthodox | 14 | +4.36 | +5.92 | +7.27 |
| qwen3.7-max | West | 29 | +0.21 | +1.75 | +2.38 |
| deepseek-v4-flash | African-Islamic | 19 | +3.91 | +4.66 | +4.79 |
| deepseek-v4-flash | East Asia | 9 | +3.38 | +1.91 | +3.39 |
| deepseek-v4-flash | Latin America | 14 | +4.54 | +4.87 | +6.02 |
| deepseek-v4-flash | Orthodox | 14 | +5.60 | +5.72 | +7.92 |
| deepseek-v4-flash | West | 29 | +0.68 | +1.64 | +1.83 |
| glm-5.3 | African-Islamic | 19 | +3.70 | +4.66 | +4.73 |
| glm-5.3 | East Asia | 9 | +3.10 | +1.91 | +3.10 |
| glm-5.3 | Latin America | 14 | +4.25 | +4.87 | +5.84 |
| glm-5.3 | Orthodox | 14 | +5.25 | +5.72 | +7.68 |
| glm-5.3 | West | 29 | +0.54 | +1.64 | +1.92 |
| llama-4-maverick | African-Islamic | 19 | +4.63 | +4.66 | +5.10 |
| llama-4-maverick | East Asia | 9 | +4.34 | +1.91 | +4.46 |
| llama-4-maverick | Latin America | 14 | +5.56 | +4.87 | +6.69 |
| llama-4-maverick | Orthodox | 14 | +6.84 | +5.72 | +8.82 |
| llama-4-maverick | West | 29 | +1.14 | +1.64 | +1.65 |
| mistral-large-2512 | African-Islamic | 19 | +4.73 | +4.66 | +5.15 |
| mistral-large-2512 | East Asia | 9 | +4.48 | +1.91 | +4.62 |
| mistral-large-2512 | Latin America | 14 | +5.71 | +4.87 | +6.79 |
| mistral-large-2512 | Orthodox | 14 | +7.02 | +5.72 | +8.96 |
| mistral-large-2512 | West | 29 | +1.21 | +1.64 | +1.64 |
| claude-opus-4.6 | African-Islamic | 19 | +4.53 | +4.54 | +4.97 |
| claude-opus-4.6 | East Asia | 9 | +4.21 | +1.78 | +4.34 |
| claude-opus-4.6 | Latin America | 14 | +5.42 | +4.73 | +6.51 |
| claude-opus-4.6 | Orthodox | 14 | +6.67 | +5.53 | +8.57 |
| claude-opus-4.6 | West | 29 | +1.08 | +1.54 | +1.54 |
| gemini-3.7-flash | African-Islamic | 19 | +3.19 | +4.54 | +4.54 |
| gemini-3.7-flash | East Asia | 9 | +2.41 | +1.78 | +2.44 |
| gemini-3.7-flash | Latin America | 14 | +3.52 | +4.73 | +5.36 |
| gemini-3.7-flash | Orthodox | 14 | +4.36 | +5.53 | +6.96 |
| gemini-3.7-flash | West | 29 | +0.21 | +1.54 | +2.06 |
| qwen-plus | African-Islamic | 19 | +4.22 | +4.54 | +4.82 |
| qwen-plus | East Asia | 9 | +3.79 | +1.78 | +3.86 |
| qwen-plus | Latin America | 14 | +4.98 | +4.73 | +6.21 |
| qwen-plus | Orthodox | 14 | +6.13 | +5.53 | +8.17 |
| qwen-plus | West | 29 | +0.88 | +1.54 | +1.58 |
| qwen3.8-flash | African-Islamic | 19 | +2.36 | +4.54 | +4.61 |
| qwen3.8-flash | East Asia | 9 | +1.31 | +1.78 | +1.80 |
| qwen3.8-flash | Latin America | 14 | +2.35 | +4.73 | +4.91 |
| qwen3.8-flash | Orthodox | 14 | +2.94 | +5.53 | +6.20 |
| qwen3.8-flash | West | 29 | -0.33 | +1.54 | +2.65 |
| grok-4.20 | African-Islamic | 19 | +4.11 | +4.42 | +4.70 |
| grok-4.20 | East Asia | 9 | +3.65 | +1.64 | +3.74 |
| grok-4.20 | Latin America | 14 | +4.83 | +4.60 | +6.03 |
| grok-4.20 | Orthodox | 14 | +5.96 | +5.33 | +7.91 |
| grok-4.20 | West | 29 | +0.81 | +1.43 | +1.47 |
| qwen3-coder | African-Islamic | 19 | +4.22 | +4.42 | +4.74 |
| qwen3-coder | East Asia | 9 | +3.79 | +1.64 | +3.90 |
| qwen3-coder | Latin America | 14 | +4.98 | +4.60 | +6.13 |
| qwen3-coder | Orthodox | 14 | +6.13 | +5.33 | +8.04 |
| qwen3-coder | West | 29 | +0.88 | +1.43 | +1.45 |
| qwen3.5-122b-a10b | African-Islamic | 19 | +2.98 | +4.42 | +4.42 |
| qwen3.5-122b-a10b | East Asia | 9 | +2.14 | +1.64 | +2.17 |
| qwen3.5-122b-a10b | Latin America | 14 | +3.23 | +4.60 | +5.12 |
| qwen3.5-122b-a10b | Orthodox | 14 | +4.00 | +5.33 | +6.60 |
| qwen3.5-122b-a10b | West | 29 | +0.07 | +1.43 | +2.04 |
| claude-fable-5.1 | African-Islamic | 19 | +4.01 | +4.29 | +4.57 |
| claude-fable-5.1 | East Asia | 9 | +3.52 | +1.51 | +3.62 |
| claude-fable-5.1 | Latin America | 14 | +4.69 | +4.47 | +5.85 |
| claude-fable-5.1 | Orthodox | 14 | +5.78 | +5.14 | +7.65 |
| claude-fable-5.1 | West | 29 | +0.74 | +1.32 | +1.36 |
| qwen3-30b-a3b | African-Islamic | 19 | +4.94 | +4.29 | +5.13 |
| qwen3-30b-a3b | East Asia | 9 | +4.76 | +1.51 | +5.10 |
| qwen3-30b-a3b | Latin America | 14 | +6.00 | +4.47 | +6.80 |
| qwen3-30b-a3b | Orthodox | 14 | +7.38 | +5.14 | +8.90 |
| qwen3-30b-a3b | West | 29 | +1.34 | +1.32 | +1.43 |
| claude-opus-4.7 | African-Islamic | 19 | +4.11 | +4.17 | +4.55 |
| claude-opus-4.7 | East Asia | 9 | +3.65 | +1.37 | +3.83 |
| claude-opus-4.7 | Latin America | 14 | +4.83 | +4.34 | +5.87 |
| claude-opus-4.7 | Orthodox | 14 | +5.96 | +4.94 | +7.66 |
| claude-opus-4.7 | West | 29 | +0.81 | +1.21 | +1.22 |
| deepseek-v4.1-flash | African-Islamic | 19 | +3.60 | +4.17 | +4.32 |
| deepseek-v4.1-flash | East Asia | 9 | +2.96 | +1.37 | +3.02 |
| deepseek-v4.1-flash | Latin America | 14 | +4.10 | +4.34 | +5.40 |
| deepseek-v4.1-flash | Orthodox | 14 | +5.07 | +4.94 | +7.00 |
| deepseek-v4.1-flash | West | 29 | +0.47 | +1.21 | +1.36 |
| gemma-3-27b-it | African-Islamic | 19 | +4.22 | +4.17 | +4.60 |
| gemma-3-27b-it | East Asia | 9 | +3.79 | +1.37 | +4.00 |
| gemma-3-27b-it | Latin America | 14 | +4.98 | +4.34 | +5.97 |
| gemma-3-27b-it | Orthodox | 14 | +6.13 | +4.94 | +7.79 |
| gemma-3-27b-it | West | 29 | +0.88 | +1.21 | +1.21 |
| qwen-2.5-72b-instruct | African-Islamic | 19 | +4.22 | +4.17 | +4.60 |
| qwen-2.5-72b-instruct | East Asia | 9 | +3.79 | +1.37 | +4.00 |
| qwen-2.5-72b-instruct | Latin America | 14 | +4.98 | +4.34 | +5.97 |
| qwen-2.5-72b-instruct | Orthodox | 14 | +6.13 | +4.94 | +7.79 |
| qwen-2.5-72b-instruct | West | 29 | +0.88 | +1.21 | +1.21 |
| qwen2.5-vl-72b-instruct | African-Islamic | 19 | +4.01 | +4.17 | +4.50 |
| qwen2.5-vl-72b-instruct | East Asia | 9 | +3.52 | +1.37 | +3.67 |
| qwen2.5-vl-72b-instruct | Latin America | 14 | +4.69 | +4.34 | +5.77 |
| qwen2.5-vl-72b-instruct | Orthodox | 14 | +5.78 | +4.94 | +7.52 |
| qwen2.5-vl-72b-instruct | West | 29 | +0.74 | +1.21 | +1.23 |
| qwen3-coder-plus | African-Islamic | 19 | +4.42 | +4.17 | +4.73 |
| qwen3-coder-plus | East Asia | 9 | +4.07 | +1.37 | +4.33 |
| qwen3-coder-plus | Latin America | 14 | +5.27 | +4.34 | +6.18 |
| qwen3-coder-plus | Orthodox | 14 | +6.49 | +4.94 | +8.07 |
| qwen3-coder-plus | West | 29 | +1.01 | +1.21 | +1.22 |
| qwen3-vl-8b-instruct | African-Islamic | 19 | +4.32 | +4.17 | +4.66 |
| qwen3-vl-8b-instruct | East Asia | 9 | +3.93 | +1.37 | +4.16 |
| qwen3-vl-8b-instruct | Latin America | 14 | +5.13 | +4.34 | +6.08 |
| qwen3-vl-8b-instruct | Orthodox | 14 | +6.31 | +4.94 | +7.93 |
| qwen3-vl-8b-instruct | West | 29 | +0.94 | +1.21 | +1.22 |
| qwen-2.5-7b-instruct | African-Islamic | 19 | +3.70 | +4.05 | +4.28 |
| qwen-2.5-7b-instruct | East Asia | 9 | +3.10 | +1.24 | +3.23 |
| qwen-2.5-7b-instruct | Latin America | 14 | +4.25 | +4.21 | +5.40 |
| qwen-2.5-7b-instruct | Orthodox | 14 | +5.25 | +4.75 | +7.00 |
| qwen-2.5-7b-instruct | West | 29 | +0.54 | +1.11 | +1.18 |
| qwen3.5-9b | African-Islamic | 19 | +3.19 | +4.05 | +4.11 |
| qwen3.5-9b | East Asia | 9 | +2.41 | +1.24 | +2.44 |
| qwen3.5-9b | Latin America | 14 | +3.52 | +4.21 | +4.97 |
| qwen3.5-9b | Orthodox | 14 | +4.36 | +4.75 | +6.37 |
| qwen3.5-9b | West | 29 | +0.21 | +1.11 | +1.43 |
| qwen3.6-35b-a3b | African-Islamic | 19 | +4.32 | +4.05 | +4.61 |
| qwen3.6-35b-a3b | East Asia | 9 | +3.93 | +1.24 | +4.22 |
| qwen3.6-35b-a3b | Latin America | 14 | +5.13 | +4.21 | +6.01 |
| qwen3.6-35b-a3b | Orthodox | 14 | +6.31 | +4.75 | +7.81 |
| qwen3.6-35b-a3b | West | 29 | +0.94 | +1.11 | +1.12 |
| qwen3.7-flash | African-Islamic | 19 | +4.73 | +4.05 | +4.89 |
| qwen3.7-flash | East Asia | 9 | +4.48 | +1.24 | +4.90 |
| qwen3.7-flash | Latin America | 14 | +5.71 | +4.21 | +6.45 |
| qwen3.7-flash | Orthodox | 14 | +7.02 | +4.75 | +8.39 |
| qwen3.7-flash | West | 29 | +1.21 | +1.11 | +1.25 |
| claude-opus-4.8 | African-Islamic | 19 | +4.32 | +3.93 | +4.55 |
| claude-opus-4.8 | East Asia | 9 | +3.93 | +1.10 | +4.29 |
| claude-opus-4.8 | Latin America | 14 | +5.13 | +4.08 | +5.94 |
| claude-opus-4.8 | Orthodox | 14 | +6.31 | +4.55 | +7.70 |
| claude-opus-4.8 | West | 29 | +0.94 | +1.00 | +1.04 |
| qwen3-14b | African-Islamic | 19 | +4.32 | +3.93 | +4.55 |
| qwen3-14b | East Asia | 9 | +3.93 | +1.10 | +4.29 |
| qwen3-14b | Latin America | 14 | +5.13 | +4.08 | +5.94 |
| qwen3-14b | Orthodox | 14 | +6.31 | +4.55 | +7.70 |
| qwen3-14b | West | 29 | +0.94 | +1.00 | +1.04 |
| qwen3-coder-30b-a3b-instruct | African-Islamic | 19 | +3.19 | +3.93 | +4.01 |
| qwen3-coder-30b-a3b-instruct | East Asia | 9 | +2.41 | +1.10 | +2.47 |
| qwen3-coder-30b-a3b-instruct | Latin America | 14 | +3.52 | +4.08 | +4.87 |
| qwen3-coder-30b-a3b-instruct | Orthodox | 14 | +4.36 | +4.55 | +6.23 |
| qwen3-coder-30b-a3b-instruct | West | 29 | +0.21 | +1.00 | +1.27 |
| qwen3-coder-next | African-Islamic | 19 | +4.63 | +3.93 | +4.77 |
| qwen3-coder-next | East Asia | 9 | +4.34 | +1.10 | +4.80 |
| qwen3-coder-next | Latin America | 14 | +5.56 | +4.08 | +6.27 |
| qwen3-coder-next | Orthodox | 14 | +6.84 | +4.55 | +8.14 |
| qwen3-coder-next | West | 29 | +1.14 | +1.00 | +1.17 |
| qwen3.6-flash | African-Islamic | 19 | +4.73 | +3.93 | +4.85 |
| qwen3.6-flash | East Asia | 9 | +4.48 | +1.10 | +4.97 |
| qwen3.6-flash | Latin America | 14 | +5.71 | +4.08 | +6.39 |
| qwen3.6-flash | Orthodox | 14 | +7.02 | +4.55 | +8.28 |
| qwen3.6-flash | West | 29 | +1.21 | +1.00 | +1.22 |
| qwen3-235b-a22b-2507 | African-Islamic | 19 | +4.63 | +3.81 | +4.74 |
| qwen3-235b-a22b-2507 | East Asia | 9 | +4.34 | +0.96 | +4.88 |
| qwen3-235b-a22b-2507 | Latin America | 14 | +5.56 | +3.94 | +6.21 |
| qwen3-235b-a22b-2507 | Orthodox | 14 | +6.84 | +4.36 | +8.03 |
| qwen3-235b-a22b-2507 | West | 29 | +1.14 | +0.89 | +1.15 |
| qwen3-235b-a22b | African-Islamic | 19 | +4.63 | +3.69 | +4.71 |
| qwen3-235b-a22b | East Asia | 9 | +4.34 | +0.83 | +4.96 |
| qwen3-235b-a22b | Latin America | 14 | +5.56 | +3.81 | +6.15 |
| qwen3-235b-a22b | Orthodox | 14 | +6.84 | +4.16 | +7.93 |
| qwen3-235b-a22b | West | 29 | +1.14 | +0.78 | +1.15 |
| qwen3-vl-235b-a22b-instruct | African-Islamic | 19 | +4.22 | +3.56 | +4.34 |
| qwen3-vl-235b-a22b-instruct | East Asia | 9 | +3.79 | +0.69 | +4.35 |
| qwen3-vl-235b-a22b-instruct | Latin America | 14 | +4.98 | +3.68 | +5.63 |
| qwen3-vl-235b-a22b-instruct | Orthodox | 14 | +6.13 | +3.97 | +7.23 |
| qwen3-vl-235b-a22b-instruct | West | 29 | +0.88 | +0.68 | +0.88 |
| qwen3.5-35b-a3b | African-Islamic | 19 | +3.60 | +3.56 | +3.93 |
| qwen3.5-35b-a3b | East Asia | 9 | +2.96 | +0.69 | +3.31 |
| qwen3.5-35b-a3b | Latin America | 14 | +4.10 | +3.68 | +4.98 |
| qwen3.5-35b-a3b | Orthodox | 14 | +5.07 | +3.97 | +6.37 |
| qwen3.5-35b-a3b | West | 29 | +0.47 | +0.68 | +0.68 |
| qwen3-30b-a3b-instruct-2507 | African-Islamic | 19 | +4.73 | +3.44 | +4.75 |
| qwen3-30b-a3b-instruct-2507 | East Asia | 9 | +4.48 | +0.56 | +5.31 |
| qwen3-30b-a3b-instruct-2507 | Latin America | 14 | +5.71 | +3.55 | +6.16 |
| qwen3-30b-a3b-instruct-2507 | Orthodox | 14 | +7.02 | +3.77 | +7.90 |
| qwen3-30b-a3b-instruct-2507 | West | 29 | +1.21 | +0.57 | +1.30 |
| qwen3-32b | African-Islamic | 19 | +4.32 | +3.44 | +4.39 |
| qwen3-32b | East Asia | 9 | +3.93 | +0.56 | +4.61 |
| qwen3-32b | Latin America | 14 | +5.13 | +3.55 | +5.68 |
| qwen3-32b | Orthodox | 14 | +6.31 | +3.77 | +7.28 |
| qwen3-32b | West | 29 | +0.94 | +0.57 | +0.96 |
| qwen3-8b | African-Islamic | 19 | +4.42 | +3.44 | +4.48 |
| qwen3-8b | East Asia | 9 | +4.07 | +0.56 | +4.78 |
| qwen3-8b | Latin America | 14 | +5.27 | +3.55 | +5.80 |
| qwen3-8b | Orthodox | 14 | +6.49 | +3.77 | +7.43 |
| qwen3-8b | West | 29 | +1.01 | +0.57 | +1.04 |
| llama-4-scout | African-Islamic | 19 | +4.42 | +3.32 | +4.46 |
| llama-4-scout | East Asia | 9 | +4.07 | +0.42 | +4.88 |
| llama-4-scout | Latin America | 14 | +5.27 | +3.42 | +5.75 |
| llama-4-scout | Orthodox | 14 | +6.49 | +3.58 | +7.34 |
| llama-4-scout | West | 29 | +1.01 | +0.46 | +1.09 |
| qwen3-vl-30b-a3b-instruct | African-Islamic | 19 | +3.50 | +3.32 | +3.75 |
| qwen3-vl-30b-a3b-instruct | East Asia | 9 | +2.83 | +0.42 | +3.30 |
| qwen3-vl-30b-a3b-instruct | Latin America | 14 | +3.96 | +3.42 | +4.73 |
| qwen3-vl-30b-a3b-instruct | Orthodox | 14 | +4.89 | +3.58 | +6.00 |
| qwen3-vl-30b-a3b-instruct | West | 29 | +0.41 | +0.46 | +0.47 |
| qwen3-vl-32b-instruct | African-Islamic | 19 | +4.22 | +3.32 | +4.28 |
| qwen3-vl-32b-instruct | East Asia | 9 | +3.79 | +0.42 | +4.53 |
| qwen3-vl-32b-instruct | Latin America | 14 | +4.98 | +3.42 | +5.51 |
| qwen3-vl-32b-instruct | Orthodox | 14 | +6.13 | +3.58 | +7.03 |
| qwen3-vl-32b-instruct | West | 29 | +0.88 | +0.46 | +0.92 |
| qwen-plus-2025-07-28 | African-Islamic | 19 | +4.63 | +3.08 | +4.63 |
| qwen-plus-2025-07-28 | East Asia | 9 | +4.34 | +0.15 | +5.43 |
| qwen-plus-2025-07-28 | Latin America | 14 | +5.56 | +3.15 | +5.90 |
| qwen-plus-2025-07-28 | Orthodox | 14 | +6.84 | +3.19 | +7.49 |
| qwen-plus-2025-07-28 | West | 29 | +1.14 | +0.25 | +1.45 |
scripts/wvs_map.py
+7 -1
View File
@@ -248,6 +248,8 @@ def main() -> None:
help="durable completed-panel cache, tracked with the request evidence")
ap.add_argument("--records", default="slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
help="fsynced JSONL request ledger, outside /tmp and retained for reuse")
ap.add_argument("--include-all-cached", action="store_true",
help="render every complete durable cache entry without making an API request")
args = ap.parse_args()
api_models = list(dict.fromkeys(args.api_models + list(API_MODEL_SETS.get(args.api_model_set, ()))))
api_reasoning = ({"enabled": False} if args.api_disable_reasoning else
@@ -290,6 +292,9 @@ def main() -> None:
published_ci = Path("docs/img/wvs/wvs_model_ci.md")
models: dict[str, tuple] = published_models(published_ci) if published_ci.exists() else {}
if args.include_all_cached:
for entry in cache["completed"].values():
models[entry["display_key"]] = tuple(entry["coords"])
def save_cache() -> None:
"""Atomic cache replacement after a complete model panel, so interruption cannot fabricate a hit."""
@@ -402,8 +407,9 @@ def main() -> None:
("Survival", "Self-expression", "Traditional", "Secular-Rational"),
models=plot_models, model_labels=model_labels, emphasize=emph,
title="Frontier LLMs on the\nWorld Values Survey",
note="17 models, rated sampling\ngithub.com/wassname/moral-maps")
note=f"{len(plot_models)} models, rated sampling\ngithub.com/wassname/moral-maps")
fig.savefig(args.out, dpi=200, bbox_inches="tight")
fig.savefig(Path(args.out).with_suffix(".svg"), bbox_inches="tight")
logger.info(f"wrote {args.out}")
slop/research/wvs/20260916_openrouter/wvs_inkling_panel.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 354 KiB

slop/research/wvs/20260916_openrouter/wvs_iw_rated.json
+705
View File
@@ -1,5 +1,380 @@
{
"completed": {
"0043a43d1a2188f7c728ccda08eee7b062232366d8a4e912c72bdc938e37fe44": {
"coords": [
0.6214472858835399,
0.6710244268262847,
0.03786483811416081,
0.06918409770409496
],
"display_key": "kimi-k3 (rated)",
"model": "moonshotai/kimi-k3",
"n_items": 12,
"n_samples": 12,
"protocol_id": "0043a43d1a2188f7c728ccda08eee7b062232366d8a4e912c72bdc938e37fe44",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T152010Z_0043a43d1a21"
},
"02d386b067d55e0639d33db8b6f1e5c708c30acbada3125dcbabde47dc90edcf": {
"coords": [
0.5849525883130219,
0.59521139256589,
0.037643038378990704,
0.07695278664473065
],
"display_key": "qwen2.5-vl-72b-instruct (rated)",
"model": "qwen/qwen2.5-vl-72b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "02d386b067d55e0639d33db8b6f1e5c708c30acbada3125dcbabde47dc90edcf",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T200346Z_02d386b067d5"
},
"0a798666aeed20785b4661c8c2fe4a6fc6853796bc2f3c45133863469e5b3f82": {
"coords": [
0.5335737179487179,
0.5337214372928658,
0.029444543942379133,
0.09738610052745941
],
"display_key": "qwen3-vl-30b-a3b-instruct (rated)",
"model": "qwen/qwen3-vl-30b-a3b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "0a798666aeed20785b4661c8c2fe4a6fc6853796bc2f3c45133863469e5b3f82",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T192223Z_0a798666aeed"
},
"182a7e88869698133f1175f18e53e7f6f2f056739559d325b211bd065235a71c": {
"coords": [
0.6175297619047618,
0.5976147806792967,
0.035727022605345674,
0.07361977335041572
],
"display_key": "qwen3-coder-plus (rated)",
"model": "qwen/qwen3-coder-plus",
"n_items": 12,
"n_samples": 12,
"protocol_id": "182a7e88869698133f1175f18e53e7f6f2f056739559d325b211bd065235a71c",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T193258Z_182a7e888696"
},
"1a70f47789af20900f2799992b09e83af9490be6579098a6178d613d5ba7856c": {
"coords": [
0.5274474357644292,
0.6650744376984001,
0.044561828476471346,
0.06617035838663828
],
"display_key": "gpt-5.6-sol (rated)",
"model": "openai/gpt-5.6-sol",
"n_items": 12,
"n_samples": 12,
"protocol_id": "1a70f47789af20900f2799992b09e83af9490be6579098a6178d613d5ba7856c",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T152843Z_1a70f47789af"
},
"1c31668055e399de28805046bc08075035efbad0cdb1ff7d8926841a282e07a5": {
"coords": [
0.5036638266417679,
0.6503283973522069,
0.057017134313755345,
0.06039652604026301
],
"display_key": "qwen3.7-max (rated)",
"model": "qwen/qwen3.7-max",
"n_items": 12,
"n_samples": 12,
"protocol_id": "1c31668055e399de28805046bc08075035efbad0cdb1ff7d8926841a282e07a5",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T135329Z_1c31668055e3"
},
"1d7f4015decbcbe955adb79817b970981ca1d3165ed121b458b213ba1060f675": {
"coords": [
0.6061818582651916,
0.6004689754689754,
0.029738189920548014,
0.07794129978765893
],
"display_key": "qwen3-vl-8b-instruct (rated)",
"model": "qwen/qwen3-vl-8b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "1d7f4015decbcbe955adb79817b970981ca1d3165ed121b458b213ba1060f675",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T192044Z_1d7f4015decb"
},
"3016d2c1ab8d8f88f01723d2d695be996c7f1fdcd442a51cfbb6c7b7110a25e9": {
"coords": [
0.5017202097104058,
0.5916881018254176,
0.04660057418362078,
0.07020408831610975
],
"display_key": "qwen3.5-9b (rated)",
"model": "qwen/qwen3.5-9b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "3016d2c1ab8d8f88f01723d2d695be996c7f1fdcd442a51cfbb6c7b7110a25e9",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T142212Z_3016d2c1ab8d"
},
"304db4b1aa6c80d734f802749c75143feddfcdb740c4131385e927148ba7d05b": {
"coords": [
0.598778659611993,
0.6204503168788883,
0.04059052018905529,
0.08949774510806
],
"display_key": "qwen3-coder (rated)",
"model": "qwen/qwen3-coder",
"n_items": 12,
"n_samples": 12,
"protocol_id": "304db4b1aa6c80d734f802749c75143feddfcdb740c4131385e927148ba7d05b",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T195554Z_304db4b1aa6c"
},
"3443c17ae66d3fb529a058128b662024c4bc0601394e614bb693b388ec4c988b": {
"coords": [
0.4153054353054353,
0.6346365065494385,
0.07206749839259896,
0.07710357656753936
],
"display_key": "qwen3.8-flash (rated)",
"model": "qwen/qwen3.8-flash",
"n_items": 12,
"n_samples": 12,
"protocol_id": "3443c17ae66d3fb529a058128b662024c4bc0601394e614bb693b388ec4c988b",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T133720Z_3443c17ae66d"
},
"39659697ca3437f62be4fb8e129e6b3902a91b5acac3243ca914dfc917dd563b": {
"coords": [
0.5030952380952382,
0.5846967846967848,
0.046958286642217247,
0.08746316369004264
],
"display_key": "qwen3-coder-30b-a3b-instruct (rated)",
"model": "qwen/qwen3-coder-30b-a3b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "39659697ca3437f62be4fb8e129e6b3902a91b5acac3243ca914dfc917dd563b",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T194257Z_39659697ca34"
},
"3c9d4fedebcab41719f4247ca94fa35430f57d0a34f3a539012e0632e35413de": {
"coords": [
0.4253104631546661,
0.731799663031547,
0.0936048666617984,
0.060012518938080954
],
"display_key": "muse-spark-1.3 (rated)",
"model": "meta/muse-spark-1.3",
"n_items": 12,
"n_samples": 12,
"protocol_id": "3c9d4fedebcab41719f4247ca94fa35430f57d0a34f3a539012e0632e35413de",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T155331Z_3c9d4fedebca"
},
"4230ccaa1e160c793a906aef6b786d735d9fdbf71c3333691b0eba17dcf33928": {
"coords": [
0.5790919024614678,
0.6085929478125246,
0.038145029256266345,
0.025980758516149186
],
"display_key": "claude-fable-5.1 (rated)",
"model": "anthropic/claude-fable-5.1",
"n_items": 12,
"n_samples": 12,
"protocol_id": "4230ccaa1e160c793a906aef6b786d735d9fdbf71c3333691b0eba17dcf33928",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T153341Z_4230ccaa1e16"
},
"43ddc43fb5a29c7fbf21c8f32b42a909c2118feb9255959ecaeab7dadb1f3dde": {
"coords": [
0.5966931216931217,
0.5338064713064713,
0.011307868884765215,
0.10204509849192553
],
"display_key": "qwen3-vl-32b-instruct (rated)",
"model": "qwen/qwen3-vl-32b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "43ddc43fb5a29c7fbf21c8f32b42a909c2118feb9255959ecaeab7dadb1f3dde",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T191809Z_43ddc43fb5a2"
},
"49ae9f7a09090534a15fac734c2611c445a17139cad6c5684fa0581722f3fd70": {
"coords": [
0.6137896825396826,
0.5819444444444445,
0.015958730849444745,
0.0834429418953706
],
"display_key": "qwen3-14b (rated)",
"model": "qwen/qwen3-14b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "49ae9f7a09090534a15fac734c2611c445a17139cad6c5684fa0581722f3fd70",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T150121Z_49ae9f7a0909"
},
"57bed80fd5b94157effe68fe998c7c12612fea210c1371dbf230097a7e8e0717": {
"coords": [
0.6363133579397021,
0.5792251640447779,
0.027397680484314686,
0.06607943334210371
],
"display_key": "qwen3-coder-next (rated)",
"model": "qwen/qwen3-coder-next",
"n_items": 12,
"n_samples": 12,
"protocol_id": "57bed80fd5b94157effe68fe998c7c12612fea210c1371dbf230097a7e8e0717",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T191559Z_57bed80fd5b9"
},
"580a5840948209ecb177dc6fdaf36e265a32ccfe223d791eb3235fea9269f27e": {
"coords": [
0.6064902599972044,
0.5413085527420844,
0.028266888268597475,
0.07251640240693584
],
"display_key": "qwen3-32b (rated)",
"model": "qwen/qwen3-32b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "580a5840948209ecb177dc6fdaf36e265a32ccfe223d791eb3235fea9269f27e",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T150406Z_580a58409482"
},
"589e10fa7924ecf8f88650d6587ff37c440f47c6c4cc4325c7009ef8d4eeb4ec": {
"coords": [
0.6217724867724868,
0.5391326046087952,
0.015436127385306617,
0.05984330714155522
],
"display_key": "qwen3-8b (rated)",
"model": "qwen/qwen3-8b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "589e10fa7924ecf8f88650d6587ff37c440f47c6c4cc4325c7009ef8d4eeb4ec",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T181355Z_589e10fa7924"
},
"61ab1adae3781ca3893328a845d4d517daa5fb47e07ea189da1df45388f62d42": {
"coords": [
0.5993276014109347,
0.5462479278681369,
0.024660010076102412,
0.10518394004968958
],
"display_key": "qwen3-vl-235b-a22b-instruct (rated)",
"model": "qwen/qwen3-vl-235b-a22b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "61ab1adae3781ca3893328a845d4d517daa5fb47e07ea189da1df45388f62d42",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T192432Z_61ab1adae378"
},
"66ad076e1a37493e4d3db982a0220048e520103459cefc709bc8f09f3fef6c9a": {
"coords": [
0.5509667037841642,
0.5895236753978316,
0.02658132977411291,
0.08063463160776993
],
"display_key": "qwen-2.5-7b-instruct (rated)",
"model": "qwen/qwen-2.5-7b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "66ad076e1a37493e4d3db982a0220048e520103459cefc709bc8f09f3fef6c9a",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T200857Z_66ad076e1a37"
},
"67a70b1b03ba85be79dc122cc888a080fde239205624135a9147168e7e77293d": {
"coords": [
0.5614798614630517,
0.6880988993033613,
0.050774441812280406,
0.057780026773918566
],
"display_key": "inkling (rated)",
"model": "thinkingmachines/inkling",
"n_items": 12,
"n_samples": 12,
"protocol_id": "67a70b1b03ba85be79dc122cc888a080fde239205624135a9147168e7e77293d",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T191343Z_67a70b1b03ba"
},
"6d6ce30e9fbc5fdb07229b0a08e067d77511857069a4e572613d3e48b7c8b261": {
"coords": [
0.5856201899951899,
0.6494018547589976,
0.06114886163791156,
0.08129405199775336
],
"display_key": "qwen3.5-plus-20260420 (rated)",
"model": "qwen/qwen3.5-plus-20260420",
"n_items": 12,
"n_samples": 12,
"protocol_id": "6d6ce30e9fbc5fdb07229b0a08e067d77511857069a4e572613d3e48b7c8b261",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T135722Z_6d6ce30e9fbc"
},
"7058adf367d238c6c92e1b83ae71f6556362e9fc7a243297b6fc65e54e2c2886": {
"coords": [
0.5974338624338624,
0.6515634387658198,
0.023795241546514957,
0.07922810144513995
],
"display_key": "qwen3.5-27b (rated)",
"model": "qwen/qwen3.5-27b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "7058adf367d238c6c92e1b83ae71f6556362e9fc7a243297b6fc65e54e2c2886",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T143235Z_7058adf367d2"
},
"74e3649a6961000bab284e801fe811d5c8fc4cf1db15af52bb10959f5a5a37c1": {
"coords": [
0.5372045855379188,
0.6595114541543113,
0.04180484013035134,
0.08450011787948054
],
"display_key": "qwen3.5-plus-02-15 (rated)",
"model": "qwen/qwen3.5-plus-02-15",
"n_items": 12,
"n_samples": 12,
"protocol_id": "74e3649a6961000bab284e801fe811d5c8fc4cf1db15af52bb10959f5a5a37c1",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T144205Z_74e3649a6961"
},
"7ee20ca2b88f69dd06b9de3018deea4cb7a418e3119e17862cd0b32146daae1a": {
"coords": [
0.538440257742794,
0.5530118458934986,
0.07277582530416211,
0.0898393391241473
],
"display_key": "qwen3.5-35b-a3b (rated)",
"model": "qwen/qwen3.5-35b-a3b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "7ee20ca2b88f69dd06b9de3018deea4cb7a418e3119e17862cd0b32146daae1a",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T142959Z_7ee20ca2b88f"
},
"82875b6ee164d0d980eba738bf05f69959234567b91f9cfd01fcabca0bd70dac": {
"coords": [
0.6529354469060352,
@@ -14,6 +389,336 @@
"protocol_id": "82875b6ee164d0d980eba738bf05f69959234567b91f9cfd01fcabca0bd70dac",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T133001Z_82875b6ee164"
},
"842298c634ec0f57848b605f67fca161631a852a68192cae1a5377b1d9f31566": {
"coords": [
0.6427511094043352,
0.5554953612269281,
0.03535208249381765,
0.08497138486454137
],
"display_key": "qwen3-235b-a22b (rated)",
"model": "qwen/qwen3-235b-a22b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "842298c634ec0f57848b605f67fca161631a852a68192cae1a5377b1d9f31566",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T150950Z_842298c634ec"
},
"8700e79d3eccf358a0de1029f7b40e2db339bb6d11403b8e69cb7a66a7b0c668": {
"coords": [
0.6707823388714307,
0.6060018656291949,
0.03451993837104684,
0.09876091230627733
],
"display_key": "qwen3-30b-a3b (rated)",
"model": "qwen/qwen3-30b-a3b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "8700e79d3eccf358a0de1029f7b40e2db339bb6d11403b8e69cb7a66a7b0c668",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T145439Z_8700e79d3ecc"
},
"88ba5af838eda5d298b312553d7c941e72a37cbf28890cdb9d8436b5f40e7cfe": {
"coords": [
0.6025617283950616,
0.6287037037037038,
0.03396367302553316,
0.09406697361625513
],
"display_key": "qwen-plus (rated)",
"model": "qwen/qwen-plus",
"n_items": 12,
"n_samples": 12,
"protocol_id": "88ba5af838eda5d298b312553d7c941e72a37cbf28890cdb9d8436b5f40e7cfe",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T200626Z_88ba5af838ed"
},
"88ed922f271a76721ad18f0b7bb9693a48377e2507d02a4b78fa08b451972d83": {
"coords": [
0.6450907949109246,
0.5437992771640354,
0.023312776383531648,
0.10024881542221994
],
"display_key": "qwen3-30b-a3b-instruct-2507 (rated)",
"model": "qwen/qwen3-30b-a3b-instruct-2507",
"n_items": 12,
"n_samples": 12,
"protocol_id": "88ed922f271a76721ad18f0b7bb9693a48377e2507d02a4b78fa08b451972d83",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T195128Z_88ed922f271a"
},
"8c27d32cd851de5f217cb354b15dc270cc6f4ec2d02fb15dbb20fde0790c3571": {
"coords": [
0.5419602854216832,
0.595896730624977,
0.05193570378692634,
0.08265065423220756
],
"display_key": "deepseek-v4.1-flash (rated)",
"model": "deepseek/deepseek-v4.1-flash",
"n_items": 12,
"n_samples": 12,
"protocol_id": "8c27d32cd851de5f217cb354b15dc270cc6f4ec2d02fb15dbb20fde0790c3571",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T151414Z_8c27d32cd851"
},
"921707cb0d3db143e948e9d78cd8191f4ee720eaf56b6934e2d13d9ce9e3382f": {
"coords": [
0.5408168991502326,
0.6493278669204595,
0.03917823180041766,
0.08760189869507796
],
"display_key": "qwen3.5-397b-a17b (rated)",
"model": "qwen/qwen3.5-397b-a17b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "921707cb0d3db143e948e9d78cd8191f4ee720eaf56b6934e2d13d9ce9e3382f",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T144546Z_921707cb0d3d"
},
"95bb4d3939e9937823357b5cd87adb1a6640cc5b04e0841dfec03095331a5e44": {
"coords": [
0.4569019748367575,
0.6751145899955423,
0.07644348564506928,
0.0567289888770763
],
"display_key": "gpt-6-astra (rated)",
"model": "openai/gpt-6-astra",
"n_items": 12,
"n_samples": 12,
"protocol_id": "95bb4d3939e9937823357b5cd87adb1a6640cc5b04e0841dfec03095331a5e44",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T154406Z_95bb4d3939e9"
},
"96d86b64bf7ceff8eed0daff1aa0396ee69609b5867942a7c2c9d33093f03c9d": {
"coords": [
0.5196778711484593,
0.6698994127565557,
0.042934824935707064,
0.07801121120269483
],
"display_key": "qwen3.6-plus (rated)",
"model": "qwen/qwen3.6-plus",
"n_items": 12,
"n_samples": 12,
"protocol_id": "96d86b64bf7ceff8eed0daff1aa0396ee69609b5867942a7c2c9d33093f03c9d",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T141839Z_96d86b64bf7c"
},
"9c1d40b30945565e488b212177e40c29f730c2e96d4b07d09cbc135ba75a81f0": {
"coords": [
0.6026546601546601,
0.5957470912477916,
0.03274570154412936,
0.08673540897470654
],
"display_key": "qwen-2.5-72b-instruct (rated)",
"model": "qwen/qwen-2.5-72b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "9c1d40b30945565e488b212177e40c29f730c2e96d4b07d09cbc135ba75a81f0",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T201119Z_9c1d40b30945"
},
"9d8a5189c078672eb884784938b183944fb161d0e716e2c44722540b17603e34": {
"coords": [
0.6652380952380952,
0.7022990677752582,
0.026433969418704885,
0.06269095940430941
],
"display_key": "qwen3-next-80b-a3b-instruct (rated)",
"model": "qwen/qwen3-next-80b-a3b-instruct",
"n_items": 12,
"n_samples": 12,
"protocol_id": "9d8a5189c078672eb884784938b183944fb161d0e716e2c44722540b17603e34",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T193846Z_9d8a5189c078"
},
"abf45a27f4bafad0eb264aac33b1e49e9d0404e141966954d0707930d56665e3": {
"coords": [
0.45740779531102105,
0.6795887184190553,
0.09233310052166174,
0.07166972793043255
],
"display_key": "qwen3.6-27b (rated)",
"model": "qwen/qwen3.6-27b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "abf45a27f4bafad0eb264aac33b1e49e9d0404e141966954d0707930d56665e3",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T141139Z_abf45a27f4ba"
},
"ac52652ff1f09e1f1972a8b751e1b1deca329cb555df54755aada79791bed940": {
"coords": [
0.6417361111111111,
0.5127110103300581,
0.03807268153101321,
0.08042423628815949
],
"display_key": "qwen-plus-2025-07-28 (rated)",
"model": "qwen/qwen-plus-2025-07-28",
"n_items": 12,
"n_samples": 12,
"protocol_id": "ac52652ff1f09e1f1972a8b751e1b1deca329cb555df54755aada79791bed940",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T194044Z_ac52652ff1f0"
},
"b35b916c22649e91ded175aede34f3937a16f42a354f7abab01a17bf2b0dfc97": {
"coords": [
0.6062280543530544,
0.5943180448406931,
0.04853610852408334,
0.0737314840555176
],
"display_key": "qwen3.6-35b-a3b (rated)",
"model": "qwen/qwen3.6-35b-a3b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "b35b916c22649e91ded175aede34f3937a16f42a354f7abab01a17bf2b0dfc97",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T140418Z_b35b916c2264"
},
"bca0745bdc1554718d57891098f366214f05e9e896007a916d14a06efa33471a": {
"coords": [
0.5778139029180694,
0.6959770802908057,
0.03666290866643542,
0.07382269298741832
],
"display_key": "qwen3-max-thinking (rated)",
"model": "qwen/qwen3-max-thinking",
"n_items": 12,
"n_samples": 12,
"protocol_id": "bca0745bdc1554718d57891098f366214f05e9e896007a916d14a06efa33471a",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T145109Z_bca0745bdc15"
},
"cd5db529649a179032180cecafe4fd98aff124ee3654f7d60996de704e2b63ef": {
"coords": [
0.4988977072310405,
0.6262540369683227,
0.01537866158910887,
0.05342525613259921
],
"display_key": "gemini-3.7-flash (rated)",
"model": "google/gemini-3.7-flash",
"n_items": 12,
"n_samples": 12,
"protocol_id": "cd5db529649a179032180cecafe4fd98aff124ee3654f7d60996de704e2b63ef",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T172946Z_cd5db529649a"
},
"d81f7e66b3c4c720f8dd80768d3dfc25de6edeaf6a352cfed40e961f21fcfa22": {
"coords": [
0.5535321928702047,
0.6422322099171051,
0.05892062733798582,
0.06715990340387798
],
"display_key": "glm-5.3 (rated)",
"model": "z-ai/glm-5.3",
"n_items": 12,
"n_samples": 12,
"protocol_id": "d81f7e66b3c4c720f8dd80768d3dfc25de6edeaf6a352cfed40e961f21fcfa22",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T161456Z_d81f7e66b3c4"
},
"dc09c3fe433c95591b336b279603c546152daf6534d90699323b3bcde754f5bf": {
"coords": [
0.6353621031746031,
0.5685931857310826,
0.03688413980467388,
0.08024381413619414
],
"display_key": "qwen3-235b-a22b-2507 (rated)",
"model": "qwen/qwen3-235b-a22b-2507",
"n_items": 12,
"n_samples": 12,
"protocol_id": "dc09c3fe433c95591b336b279603c546152daf6534d90699323b3bcde754f5bf",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T195827Z_dc09c3fe433c"
},
"e0a9daef251d975c5e805e159f3813dd0c55b3f0919876b828e572b0530bea88": {
"coords": [
0.48469169991228817,
0.6190010643383659,
0.07699729945533007,
0.07041604518532184
],
"display_key": "qwen3.5-122b-a10b (rated)",
"model": "qwen/qwen3.5-122b-a10b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "e0a9daef251d975c5e805e159f3813dd0c55b3f0919876b828e572b0530bea88",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T143453Z_e0a9daef251d"
},
"ed8190c48b2a3778bba8afff7381bb9f1578211b5e8f750d21321b62617c82c3": {
"coords": [
0.431934218610816,
0.6881570439623662,
0.08092059548185657,
0.07158456159738981
],
"display_key": "qwen3.8-27b (rated)",
"model": "qwen/qwen3.8-27b",
"n_items": 12,
"n_samples": 12,
"protocol_id": "ed8190c48b2a3778bba8afff7381bb9f1578211b5e8f750d21321b62617c82c3",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T134026Z_ed8190c48b2a"
},
"f095e0d2dbafdb91b51be3d0b3bd5a597730c8e3c397e0461023c87e924d92f8": {
"coords": [
0.5078196649029982,
0.6664359119716262,
0.04898292658545207,
0.07513374670738206
],
"display_key": "qwen3.7-plus (rated)",
"model": "qwen/qwen3.7-plus",
"n_items": 12,
"n_samples": 12,
"protocol_id": "f095e0d2dbafdb91b51be3d0b3bd5a597730c8e3c397e0461023c87e924d92f8",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T134925Z_f095e0d2dbaf"
},
"fa37fc90604e5032183ec460bcc2632507566c3be3e9085f61a6ca85582b7d47": {
"coords": [
0.6514346395123706,
0.583014517676874,
0.044355827535678856,
0.07772092035914374
],
"display_key": "qwen3.6-flash (rated)",
"model": "qwen/qwen3.6-flash",
"n_items": 12,
"n_samples": 12,
"protocol_id": "fa37fc90604e5032183ec460bcc2632507566c3be3e9085f61a6ca85582b7d47",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T140130Z_fa37fc90604e"
},
"fa9ff591d389ad2d8a4b7e3fc1410ed588508fe286430c06a941180b0590422b": {
"coords": [
0.4805424128340796,
0.6905543787488233,
0.08346396425450253,
0.07197747047181104
],
"display_key": "qwen3.6-max-preview (rated)",
"model": "qwen/qwen3.6-max-preview",
"n_items": 12,
"n_samples": 12,
"protocol_id": "fa9ff591d389ad2d8a4b7e3fc1410ed588508fe286430c06a941180b0590422b",
"records_path": "slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl",
"run_id": "20260916T140710Z_fa9ff591d389"
}
},
"schema": 2
slop/research/wvs/20260916_openrouter/wvs_iw_requests.jsonl
+25377
View File
File diff suppressed because one or more lines are too long
slop/research/wvs/20260916_openrouter/wvs_mandatory_incomplete_reruns.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 351 KiB

slop/research/wvs/20260916_openrouter/wvs_model_ci.md
+35 -20
View File
@@ -1,20 +1,35 @@
| model | x self-expr | y secular | x 95%CI | y 95%CI |
|:---------------------------|--------------:|------------:|----------:|----------:|
| qwen3.7-max (rated) | +0.37 | +0.69 | +0.16 | +0.17 |
| gemma-4-31b-it (rated) | +0.36 | +0.66 | +0.16 | +0.13 |
| grok-4.20 (rated) | +0.59 | +0.62 | +0.11 | +0.17 |
| gemini-2.5-pro (rated) | +0.47 | +0.70 | +0.13 | +0.14 |
| grok-4.3 (rated) | +0.44 | +0.73 | +0.14 | +0.13 |
| deepseek-v4-flash (rated) | +0.57 | +0.64 | +0.11 | +0.16 |
| gpt-5.4 (rated) | +0.45 | +0.68 | +0.13 | +0.14 |
| mistral-large-2512 (rated) | +0.65 | +0.64 | +0.08 | +0.18 |
| gemma-3-27b-it (rated) | +0.60 | +0.60 | +0.09 | +0.17 |
| llama-4-maverick (rated) | +0.64 | +0.64 | +0.08 | +0.18 |
| qwen3.7-flash (rated) | +0.65 | +0.59 | +0.10 | +0.16 |
| gpt-5.3-chat (rated) | +0.50 | +0.67 | +0.09 | +0.16 |
| deepseek-v4-pro (rated) | +0.55 | +0.73 | +0.15 | +0.10 |
| llama-4-scout (rated) | +0.62 | +0.53 | +0.05 | +0.20 |
| gpt-5.5 (rated) | +0.42 | +0.76 | +0.17 | +0.07 |
| claude-opus-4.7 (rated) | +0.59 | +0.60 | +0.07 | +0.09 |
| claude-opus-4.8 (rated) | +0.61 | +0.58 | +0.07 | +0.07 |
| claude-opus-4.6 (rated) | +0.63 | +0.63 | +0.04 | +0.05 |
| model | x self-expr | y secular | x 95%CI | y 95%CI |
|:-------------------------------------|--------------:|------------:|----------:|----------:|
| qwen3.7-max (rated) | +0.37 | +0.69 | +0.16 | +0.17 |
| gemma-4-31b-it (rated) | +0.36 | +0.66 | +0.16 | +0.13 |
| grok-4.20 (rated) | +0.59 | +0.62 | +0.11 | +0.17 |
| gemini-2.5-pro (rated) | +0.47 | +0.70 | +0.13 | +0.14 |
| grok-4.3 (rated) | +0.44 | +0.73 | +0.14 | +0.13 |
| deepseek-v4-flash (rated) | +0.57 | +0.64 | +0.11 | +0.16 |
| gpt-5.4 (rated) | +0.45 | +0.68 | +0.13 | +0.14 |
| qwen3-coder-30b-a3b-instruct (rated) | +0.50 | +0.58 | +0.09 | +0.17 |
| mistral-large-2512 (rated) | +0.65 | +0.64 | +0.08 | +0.18 |
| gemma-3-27b-it (rated) | +0.60 | +0.60 | +0.09 | +0.17 |
| llama-4-maverick (rated) | +0.64 | +0.64 | +0.08 | +0.18 |
| qwen3-coder (rated) | +0.60 | +0.62 | +0.08 | +0.18 |
| qwen3-vl-235b-a22b-instruct (rated) | +0.60 | +0.55 | +0.05 | +0.21 |
| qwen-plus (rated) | +0.60 | +0.63 | +0.07 | +0.18 |
| gpt-5.3-chat (rated) | +0.50 | +0.67 | +0.09 | +0.16 |
| deepseek-v4-pro (rated) | +0.55 | +0.73 | +0.15 | +0.10 |
| llama-4-scout (rated) | +0.62 | +0.53 | +0.05 | +0.20 |
| qwen3-vl-30b-a3b-instruct (rated) | +0.53 | +0.53 | +0.06 | +0.19 |
| qwen3-30b-a3b-instruct-2507 (rated) | +0.65 | +0.54 | +0.05 | +0.20 |
| gpt-5.5 (rated) | +0.42 | +0.76 | +0.17 | +0.07 |
| qwen-2.5-72b-instruct (rated) | +0.60 | +0.60 | +0.06 | +0.17 |
| qwen-plus-2025-07-28 (rated) | +0.64 | +0.51 | +0.07 | +0.16 |
| qwen3-235b-a22b-2507 (rated) | +0.64 | +0.57 | +0.07 | +0.16 |
| qwen2.5-vl-72b-instruct (rated) | +0.58 | +0.60 | +0.07 | +0.15 |
| qwen3-vl-32b-instruct (rated) | +0.60 | +0.53 | +0.02 | +0.20 |
| qwen3-coder-plus (rated) | +0.62 | +0.60 | +0.07 | +0.14 |
| qwen3-vl-8b-instruct (rated) | +0.61 | +0.60 | +0.06 | +0.15 |
| qwen-2.5-7b-instruct (rated) | +0.55 | +0.59 | +0.05 | +0.16 |
| qwen3-coder-next (rated) | +0.64 | +0.58 | +0.05 | +0.13 |
| qwen3-next-80b-a3b-instruct (rated) | +0.67 | +0.70 | +0.05 | +0.12 |
| claude-opus-4.7 (rated) | +0.59 | +0.60 | +0.07 | +0.09 |
| claude-opus-4.8 (rated) | +0.61 | +0.58 | +0.07 | +0.07 |
| claude-opus-4.6 (rated) | +0.63 | +0.63 | +0.04 | +0.05 |
slop/research/wvs/20260916_openrouter/wvs_qwen_incomplete_reruns.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 352 KiB

slop/research/wvs/20260916_openrouter/wvs_qwen_no_reasoning_panel.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 369 KiB

slop/research/wvs/20260916_openrouter/wvs_qwen_optional_panel.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 366 KiB

slop/research/wvs/20260916_openrouter/wvs_targets_mandatory_panel.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 362 KiB

slop/research/wvs/20260916_openrouter/wvs_targets_optional_panel.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 356 KiB