wassname/ml_debug
1
0
Fork 0
mirror of https://github.com/wassname/ml_debug.git synced 2026-08-25 11:21:20 +08:00
Code Issues Packages Projects Releases Wiki Activity

readme: say plainly that only the cheap model reads the document

Browse Source
This commit is contained in:
wassname
2026-08-16 07:15:47 +08:00
parent ab9779ec00
commit 8ba59c54b8
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -2
View File
@@ -35,9 +35,10 @@ For scale, changing the judge moves a score by 0.04 on average, so the effect is
the noise. Two questions cite this repo as their source, and dropping them raises the effect rather
than lowering it, so the model is not just reading the answer.
Put next to the frontier models on the same questions, the document closes most of the gap:
Only deepseek-v4-flash-0731 reads the document. The frontier models below answer the same questions
with no document, as they normally do, and the document closes most of the distance to them:
| against | questions in common | bare | with SKILL.md | the other model | gap closed |
| frontier model, no document | questions in common | flash, no document | flash, with SKILL.md | the frontier model | distance closed |
| --- | --- | --- | --- | --- | --- |
| gpt-5.6-sol | 9 | +0.540 | +0.678 | +0.774 | 59% |
| glm-5.2, the top of the table | 11 | +0.563 | +0.698 | +0.694 | 103% |