From 8ba59c54b89dd67fa0125c64dedfc31f659209ee Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Sun, 16 Aug 2026 07:15:47 +0800 Subject: [PATCH] readme: say plainly that only the cheap model reads the document --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2ad7d11..11d7e60 100644 --- a/README.md +++ b/README.md @@ -35,9 +35,10 @@ For scale, changing the judge moves a score by 0.04 on average, so the effect is the noise. Two questions cite this repo as their source, and dropping them raises the effect rather than lowering it, so the model is not just reading the answer. -Put next to the frontier models on the same questions, the document closes most of the gap: +Only deepseek-v4-flash-0731 reads the document. The frontier models below answer the same questions +with no document, as they normally do, and the document closes most of the distance to them: -| against | questions in common | bare | with SKILL.md | the other model | gap closed | +| frontier model, no document | questions in common | flash, no document | flash, with SKILL.md | the frontier model | distance closed | | --- | --- | --- | --- | --- | --- | | gpt-5.6-sol | 9 | +0.540 | +0.678 | +0.774 | 59% | | glm-5.2, the top of the table | 11 | +0.563 | +0.698 | +0.694 | 103% |