wassname/machiavelli_deep_value
1
0
Fork 0
mirror of https://github.com/wassname/machiavelli_deep_value.git synced 2026-08-20 12:30:18 +08:00
Code Issues Packages Projects Releases Wiki Activity

Focus the motive headline on Qwen models

Browse Source
This commit is contained in:
wassname
2026-07-30 15:29:38 +08:00
parent fe2e63b6fa
commit aebb86ce25
2 changed files with 31 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -4
View File
@@ -38,11 +38,10 @@ columns, QA flags, and limitations.
A fixed Gemma 4 31B judge compared the assigned motives in both account orders.
With no rubric, crossed-set accuracy declines slightly across the five Qwen
explanation models; Kimi K3 is shown separately because it is from another model
family. This describes the judge finding the generated accounts harder to
classify, not stronger models concealing motives better.
explanation models. This describes the judge finding the generated accounts
harder to classify, not stronger models concealing motives better.
![No-rubric assigned-motive accuracy by explanation-model capability](results/no_rubric_motive_by_agent.svg)
![No-rubric assigned-motive accuracy across five Qwen explanation models](results/no_rubric_motive_by_agent.svg)
## Reproduce it
results/no_rubric_motive_by_agent.svg
+28 -41
View File
@@ -1,54 +1,41 @@
<svg xmlns="http://www.w3.org/2000/svg" width="760" height="440" viewBox="0 0 760 440">
<rect width="760" height="440" fill="white"/>
<svg xmlns="http://www.w3.org/2000/svg" width="760" height="420" viewBox="0 0 760 420">
<rect width="760" height="420" fill="white"/>
<g font-family="sans-serif" fill="#222">
<text x="380" y="28" text-anchor="middle" font-size="18">Assigned-motive accuracy by explanation model</text>
<text x="380" y="49" text-anchor="middle" font-size="11" fill="#555">no rubric; fixed Gemma 4 31B judge; crossed action-harm set</text>
<text x="380" y="70" text-anchor="middle" font-size="10" fill="#555">circles: Qwen 3.5; square: Qwen 3.7 Max; diamond: Kimi K3</text>
<text x="380" y="30" text-anchor="middle" font-size="18">Assigned-motive accuracy across five Qwen models</text>
<text x="380" y="52" text-anchor="middle" font-size="11" fill="#555">no rubric; fixed Gemma 4 31B judge; crossed action-harm set</text>
<line x1="70" y1="340" x2="720" y2="340" stroke="#777" stroke-width="0.8"/>
<line x1="70" y1="90" x2="70" y2="340" stroke="#777" stroke-width="0.8"/>
<line x1="70" y1="215" x2="720" y2="215" stroke="#999" stroke-width="1" stroke-dasharray="5 4"/>
<text x="716" y="209" text-anchor="end" font-size="10" fill="#666">chance</text>
<line x1="70" y1="330" x2="710" y2="330" stroke="#777" stroke-width="0.8"/>
<line x1="70" y1="85" x2="70" y2="330" stroke="#777" stroke-width="0.8"/>
<line x1="70" y1="207.5" x2="710" y2="207.5" stroke="#999" stroke-width="1" stroke-dasharray="5 4"/>
<text x="706" y="201.5" text-anchor="end" font-size="10" fill="#666">chance</text>
<text x="58" y="344" text-anchor="end" font-size="10" fill="#555">0.25</text>
<text x="58" y="281.5" text-anchor="end" font-size="10" fill="#555">0.375</text>
<text x="58" y="219" text-anchor="end" font-size="10" fill="#555">0.50</text>
<text x="58" y="156.5" text-anchor="end" font-size="10" fill="#555">0.625</text>
<text x="58" y="94" text-anchor="end" font-size="10" fill="#555">0.75</text>
<text x="17" y="215" text-anchor="middle" font-size="12" transform="rotate(-90 17 215)">crossed-set accuracy</text>
<text x="58" y="334" text-anchor="end" font-size="10" fill="#555">0.25</text>
<text x="58" y="272.75" text-anchor="end" font-size="10" fill="#555">0.375</text>
<text x="58" y="211.5" text-anchor="end" font-size="10" fill="#555">0.50</text>
<text x="58" y="150.25" text-anchor="end" font-size="10" fill="#555">0.625</text>
<text x="58" y="89" text-anchor="end" font-size="10" fill="#555">0.75</text>
<text x="17" y="207.5" text-anchor="middle" font-size="12" transform="rotate(-90 17 207.5)">crossed-set accuracy</text>
<line x1="70" y1="177.5" x2="514.4" y2="215.5" stroke="#333" stroke-width="1.6"/>
<text x="500" y="205" text-anchor="end" font-size="10" fill="#444">Qwen fit: -0.003 / AA point</text>
<g stroke="#555" stroke-opacity="0.45" stroke-width="1.2">
<line x1="70" y1="102" x2="70" y2="227.5"/>
<line x1="212.2" y1="97.5" x2="212.2" y2="255"/>
<line x1="265.6" y1="129" x2="265.6" y2="299"/>
<line x1="301.1" y1="149" x2="301.1" y2="311"/>
<line x1="514.4" y1="118" x2="514.4" y2="282.5"/>
<line x1="710" y1="109.5" x2="710" y2="276"/>
</g>
<line x1="70" y1="171.7" x2="710" y2="208" stroke="#333" stroke-width="1.6"/>
<text x="700" y="224" text-anchor="end" font-size="10" fill="#444">fit: -0.003 accuracy / AA point</text>
<g fill="#333">
<circle cx="70" cy="165" r="4"/>
<circle cx="212.2" cy="172" r="4"/>
<circle cx="265.6" cy="211.5" r="4"/>
<circle cx="301.1" cy="229.5" r="4"/>
<rect x="510.4" y="193" width="8" height="8"/>
<path d="M 710 184 L 716 190 L 710 196 L 704 190 Z"/>
<circle cx="70" cy="158.5" r="4.5"/>
<circle cx="274.8" cy="165.4" r="4.5"/>
<circle cx="351.6" cy="204.1" r="4.5"/>
<circle cx="402.8" cy="221.7" r="4.5"/>
<rect x="705.5" y="185.4" width="9" height="9"/>
</g>
<g font-size="10" fill="#555">
<text x="70" y="357" text-anchor="middle">21</text>
<text x="212.2" y="357" text-anchor="middle">29</text>
<text x="265.6" y="357" text-anchor="middle">32</text>
<text x="301.1" y="357" text-anchor="middle">34</text>
<text x="514.4" y="357" text-anchor="middle">46</text>
<text x="710" y="357" text-anchor="middle">57</text>
<text x="70" y="347" text-anchor="middle">21</text>
<text x="274.8" y="347" text-anchor="middle">29</text>
<text x="351.6" y="347" text-anchor="middle">32</text>
<text x="402.8" y="347" text-anchor="middle">34</text>
<text x="710" y="347" text-anchor="middle">46</text>
</g>
<text x="380" y="398" text-anchor="middle" font-size="11" fill="#555">Artificial Analysis score</text>
<text x="380" y="422" text-anchor="middle" font-size="10" fill="#666">points: model means; bars: 95% scenario bootstrap; line: Qwen-only linear fit</text>
<text x="380" y="378" text-anchor="middle" font-size="11" fill="#555">Artificial Analysis score</text>
<text x="380" y="402" text-anchor="middle" font-size="10" fill="#666">circles: Qwen 3.5; square: Qwen 3.7 Max; line: five-model linear fit</text>
</g>
</svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 3.0 KiB

After

Width:  |  Height:  |  Size: 2.4 KiB