validation: 82.6% is irreproducible -- its OWN code gives 0.780 on Qwen3-4B

Ran the exact 2026-05-08 eval (worktree at commit b20ec56, word readout) on
Qwen3-4B: top1 0.780, not 0.826. Every eval version agrees on ~0.78 (digit 0.773,
word-current 0.788, word-original 0.780). The 82.6% was a stale/erroneous table
entry, not a target this model reaches under any pipeline. Canonical value 0.773.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>

README.md

@@ -188,16 +188,15 @@ Agreement, Qwen3-4B on `classic`:
 Per-class top-1 recall is uneven (Care/Fairness/Sanctity ~1.0; Loyalty 0.56, Liberty 0.53). The
 weak spots match the usual MFT pattern: binding foundations cluster, liberty overlaps care/harm.
 
-An earlier build reported 82.6% on the same model. That number used the old readout that scored the
-first token of each foundation *word*; the canonical eval now scores the option *index digit* instead,
-deliberately, because the words tokenize into uneven first pieces (`fair`, `loy`, `san`) whose unequal
-priors leaked into the softmax (see `guided.py`). The digit readout is less biased but reads ~5 points
-lower top-1. No lever in the current code recovers the 82.6%: think budget (0.72 at 64, 0.77 at 256,
-512 collapses), BMA over 8 stochastic thinks (0.72), model scale (Qwen3-8B also 0.773), and even
-reinstating the old word-first-token gather (0.788, `scripts/probe_word_readout.py`). So the 82.6%
-came from the broader 2026-05-08 eval pipeline, not the readout alone, and the current rigorous eval
-tops out around 0.77-0.79. 0.773 is the canonical (digit) number; the gap to 0.826 is a superseded
-pipeline, not a model or config shortfall.
+An earlier build's table reported 82.6% here, but that number does not reproduce, even by running its
+own original code. Checking out the exact 2026-05-08 eval (commit `b20ec56`, the word-first-token
+readout) and running it on this model gives top-1 0.780, not 0.826. Every other route agrees on ~0.78:
+the canonical digit readout 0.773, the word readout in the current core 0.788
+(`scripts/probe_word_readout.py`), think budget 0.72-0.77, BMA 0.72, and Qwen3-8B 0.773. So this
+model's MFV top-1 is ~0.78 robustly; the 82.6% was a stale/erroneous table entry, not a target the
+model reaches under any eval version. The canonical eval scores the option *index digit* (not the
+foundation word) on purpose: the words tokenize into uneven first pieces (`fair`, `loy`, `san`) whose
+unequal priors leak into the softmax (see `guided.py`).
 
 Sensitivity to steering: a small calibrated vector registers as a shift in `Δ log p[f]`. On the
 Qwen3-4B showcase the base MFV readout is coherent (`emitted_close` 4/264, `pmass` >= 0.985, top-1

docs/RESEARCH_JOURNAL.md

@@ -707,8 +707,11 @@ that scored the first token of each foundation *word*; the canonical eval now
 scores the option *index digit* (deliberately, to drop the uneven-first-piece
 word prior, guided.py:345). The digit readout reads ~5 pts lower. Config levers
 within the digit readout do NOT recover it: top-1 0.72 (think 64), 0.77 (256),
-0.72 (BMA n_samples=8, temp 0.7) -- so 0.773 is the honest ceiling for the
-debiased eval. Reaching 0.83 needs the superseded word readout, which would be a
-method change for the sake of a number (research poison), so 0.773 stands as the
-correct current value. Switching the showcase to Qwen3-4B (the validation model)
-gave the cleaner, fully-coherent, bidirectional result.
+0.72 (BMA n_samples=8, temp 0.7). UPDATE: the 82.6% does not reproduce even by
+running its OWN original code -- a git worktree at commit b20ec56 (the 2026-05-08
+word-readout eval) on Qwen3-4B gives top1 0.780 (job 233), and the word readout in
+the current core gives 0.788 (probe_word_readout.py). So this model's MFV top-1 is
+~0.78 across every eval version (digit 0.773 / word-current 0.788 / word-original
+0.780); the 82.6% was a stale/erroneous table entry, not a reachable target. 0.773
+is the canonical (digit) value. Switching the showcase to Qwen3-4B gave the
+cleaner, fully-coherent, bidirectional result.