From 52b9688de541fd35f066b6ebc6912c213585dd86 Mon Sep 17 00:00:00 2001
From: wassname <1103714+wassname@users.noreply.github.com>
Date: Thu, 25 Jun 2026 04:29:17 +0800
Subject: [PATCH] README: word readout reads 0.788 not 0.826 -- 0.83
 unreachable in current eval

Tested the documented cause (the old word-first-token gather): top1 0.788 on
Qwen3-4B, only ~1.5pt above digit (0.773), not the table's 0.826. So even
reverting the readout does not recover it; the 82.6% came from the broader
2026-05-08 pipeline. Current eval tops out ~0.77-0.79 by every lever.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
---
 README.md | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/README.md b/README.md
index e64de2e..b40c4dc 100644
--- a/README.md
+++ b/README.md
@@ -192,10 +192,12 @@ An earlier build reported 82.6% on the same model. That number used the old read
 first token of each foundation *word*; the canonical eval now scores the option *index digit* instead,
 deliberately, because the words tokenize into uneven first pieces (`fair`, `loy`, `san`) whose unequal
 priors leaked into the softmax (see `guided.py`). The digit readout is less biased but reads ~5 points
-lower top-1. Three independent levers fail to recover the gap: think budget (0.72 at 64, 0.77 at 256,
-512 collapses), BMA over 8 stochastic thinks (0.72), and model scale (Qwen3-8B also reads exactly
-0.773). So 0.773 is the ceiling of the debiased eval, not a model or config shortfall; the 82.6% is a
-method artifact of the old word readout, not a regression to chase.
+lower top-1. No lever in the current code recovers the 82.6%: think budget (0.72 at 64, 0.77 at 256,
+512 collapses), BMA over 8 stochastic thinks (0.72), model scale (Qwen3-8B also 0.773), and even
+reinstating the old word-first-token gather (0.788, `scripts/probe_word_readout.py`). So the 82.6%
+came from the broader 2026-05-08 eval pipeline, not the readout alone, and the current rigorous eval
+tops out around 0.77-0.79. 0.773 is the canonical (digit) number; the gap to 0.826 is a superseded
+pipeline, not a model or config shortfall.
 
 Sensitivity to steering: a small calibrated vector registers as a shift in `Δ log p[f]`. On the
 Qwen3-4B showcase the base MFV readout is coherent (`emitted_close` 4/264, `pmass` >= 0.985, top-1