diff --git a/refs/llm_judges.md b/refs/llm_judges.md index 814ca0f..8f003d1 100644 --- a/refs/llm_judges.md +++ b/refs/llm_judges.md @@ -4,21 +4,21 @@ Appendix to the [ML Debugging skill](../SKILL.md). When an LLM-judged eval looks ## Numbers worth knowing (2026) -Operational rules of thumb; verbatim quotes, sources, and calibration in [llm_judge_litreview.md](llm_judge_litreview.md). +Operational rules of thumb, anchored in verbatim quotes; full passages, sources, and calibration in [llm_judge_litreview.md](llm_judge_litreview.md). Source type is stated so you can weight it: independent benchmarks and peer-reviewed studies carry more than single-group preprints. -- Order bias is still large in 2026. Averaged across 36 models, a judge flips its own verdict about 43% of the time when you just swap which answer is shown first, and it flips more when the two answers are close in quality (exactly the hard cases you care about). Even a frontier judge can flip ~66%, so bigger is not safe. The fix is cheap: run each pair in both orders and average, and treat a high flip rate as a broken test, not a result. +- Swap the order and average, on every judge. Position bias is still large in 2026: an independently-run public benchmark of 36 models reports "the model-average order-flip rate is 43.0%, and the median model flips in 41.3% of decisive two-view cases",[^lechmazur] and a peer-reviewed study of 150k+ judgements finds it concentrates on the hard cases, since position bias "is strongly affected by the quality gap between solutions".[^shi] The two agree from independent methods, so it is very probable this holds for your judge; even a frontier model flips ~66%, making a bigger judge a weak prior at best, not a fix. Running both orders and averaging is the fix, and a high flip rate means the test is broken, not decided. -- Self-preference shrinks with model size. On a bias score that nets out real quality, a 0.5B judge inflated its own outputs by ~42% while a 14B judge did so by only ~2%. Turning on reasoning does not remove it. So never let a small model grade its own family's outputs, and prefer a larger, different-family judge. +- Don't let a small model grade its own family. A peer-reviewed EMNLP study, on a metric that nets out genuine quality, reports "the DBG score of Qwen2.5-0.5B-Instruct is 41.7%. In contrast, the DBG score of Qwen2.5-14B-Instruct is only 2.1%",[^selfpref] and finds reasoning does not rescue it, "the self-preference bias in reasoning models is not necessarily less significant than the bias found in language models".[^selfpref] The shrink-with-size direction is probable but partly confounded with capability (the authors credit better instruction-following), so treat "use a larger, different-family judge than the model under test" as the safe rule rather than size being the true lever. -- A reasoning judge grades better but is not less biased. Switching a judge to a reasoning model buys roughly +8 F1 on reasoning-heavy grading (verifying evidence, catching errors), but it keeps the same length, position, and style biases. Use one when the grading itself needs reasoning; do not expect it to fix the biases above. +- A reasoning judge grades better, but probably not less biased. One group's RL-trained judge reports that judging is "inherently reasoning-intensive ... it requires verifying evidence, identifying errors, and justifying decisions", and that such judges "consistently outperform SFT-tuned baselines in the same size ... and even surpass state-of-the-art reasoning models".[^judgelrm] This is a single-group result, not yet independently replicated, so weight it as suggestive: reach for a reasoning judge when the grading itself needs that work, but expect it to keep the length, position, and style biases above. -- More thinking tokens help only up to a point. In one controlled study accuracy rose (82% to 87%) as thinking grew to ~1.1k tokens, then fell as thinking kept growing, because extra tokens add variance, not insight. Match the reasoning budget to task difficulty: on easy items, cap thinking low and spend the saved budget on more repeat passes instead. +- Match the thinking budget to task difficulty. One controlled study finds more reasoning helps only up to a point: "accuracy increases from 82.2% to 87.3% as the average number of thinking tokens increases from 385 to 1100. However, this trend does not continue indefinitely",[^overthink] and it declines as tokens grow further, because extra thinking adds variance, not insight. The exact peak is setup-specific, but the non-monotonic shape is likely general, so on easy items cap thinking low and spend the saved budget on repeat passes instead. -- Know what your reasoning-effort setting actually buys. "High effort" is not a fixed number: litellm's stock default caps it at 4096 thinking tokens, but the CAIS simple-evals harness overrides it to 24576 (~6x more). If you set effort=high without checking which mapping is live, you may be silently truncating the judge's reasoning and scoring a cut-off verdict as a real one. +- Check what "high effort" actually buys before trusting it. This one is certain, it is just what the harnesses ship: litellm's stock default caps high reasoning at 4096 tokens, while CAIS's simple-evals overrides it to "DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET=24576" (~6x more).[^cais] Set it explicitly, or you may silently truncate the judge and score a cut-off verdict as a real one. -- Repeat passes stop helping early. Self-consistency (sample the judge N times, take the majority) plateaus by about N=10-15 on strong 2026 models, down from ~40 in the original 2022 work, and accuracy can even drop past the plateau as noise creeps in. So N=4-10 repeats is plenty to measure a judge's stability; going higher mostly costs tokens. +- A few repeats measure stability; many just cost tokens. A single-author preprint (treat as suggestive) reports self-consistency "gains plateau early and, in some configurations, decline at high sample counts",[^loo] with the plateau now around N=10-15 on strong 2026 models, down from ~40 in the widely-cited 2022 work. N=4-10 repeats is very probably enough for the repeat-variance check below; going higher mostly buys noise. -- Long inputs quietly degrade the judge (context rot). Reliability starts dropping well before the context window is full: on a hard retrieval test most models fall below half their short-context score by 32K tokens. Put the rubric and the answer-under-test at the start or end of the prompt, never buried in the middle of a long reference block, where models attend least. +- Keep the judge's inputs short and edge-loaded. A peer-reviewed long-context test that strips literal keyword cues reports "At 32K, for instance, 11 models drop below 50% of their strong short-length baselines",[^nolima] and the middle-of-context penalty is well replicated across model families. So it is probable your judge degrades on long inputs well before the window fills; put the rubric and answer-under-test at the start or end of the prompt, never buried in the middle where models attend least. ## The measured biases @@ -105,3 +105,11 @@ For a worked example, wassname has a ~300-line async OpenRouter judge (WIP) that [^survey]: Gu et al., "A Survey on LLM-as-a-Judge" (2024) — https://arxiv.org/abs/2411.15594 (broad survey of methods, biases, and reliability; complements Yan's practitioner review) [^doddapaneni]: Doddapaneni, Khan, Verma, Khapra, "Finding Blind Spots in Evaluator LLMs with Interpretable Checklists" (2024) — https://arxiv.org/abs/2406.13439 (evaluator LLMs missed injected quality drops in >50% of cases) ([cache](../docs/evidence/llm_judge_biases.md)) [^shankar]: Shankar, Zamfirescu-Pereira, Hartmann, Parameswaran, Arawjo, "Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences" (2024) — https://arxiv.org/abs/2404.12272 (criteria drift; LLM evaluators need human validation) ([cache](../docs/evidence/llm_judge_biases.md)) +[^lechmazur]: Lech Mazur, position_bias benchmark — https://github.com/lechmazur/position_bias (independent, outsider-run swapped-order harness; 193 pairs, 36 models, 2026-era; strong trust signal, but a solo-run leaderboard not a paper) ([litreview](llm_judge_litreview.md)) +[^shi]: Shi et al., "Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge" (IJCNLP-AACL 2025) — https://arxiv.org/abs/2406.07791 (peer-reviewed; 150k+ instances, 15 judges; bias worsens as the answer quality gap shrinks) ([litreview](llm_judge_litreview.md)) +[^selfpref]: Chen et al., "Beyond the Surface: Measuring Self-Preference in LLM Judgments" (EMNLP 2025 main) — https://arxiv.org/abs/2506.02592 (peer-reviewed; DBG nets out quality; larger judges less self-biased, though authors credit capability; reasoning models still biased) ([litreview](llm_judge_litreview.md)) +[^judgelrm]: Chen et al., "JudgeLRM: Large Reasoning Models as a Judge" (2025) — https://arxiv.org/abs/2504.00050 (single-group preprint, not independently replicated; RL-trained reasoning judges beat same-size SFT, ~+8 F1 headline in body) ([litreview](llm_judge_litreview.md)) +[^overthink]: Ghosal et al., "Does Thinking More always Help? ... Mirage of Test-Time Scaling in Reasoning Models" (2025) — https://arxiv.org/abs/2506.04210 (preprint; one controlled study, accuracy-vs-thinking-token curve is non-monotonic, peak setup-specific) ([litreview](llm_judge_litreview.md)) +[^cais]: CAIS simple-evals .env.example vs litellm constants.py (fetched 2026-07, directly verifiable config) — https://github.com/centerforaisafety/simple-evals/blob/main/.env.example (effort high=24576/med=8192/low=1024, overriding litellm stock 4096/2048/1024) +[^loo]: Loo, "Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs" (2025) — https://arxiv.org/abs/2511.00751 (single-author preprint, low citation signal; plateau ~N=10-15 on modern models, can decline past it) ([litreview](llm_judge_litreview.md)) +[^nolima]: Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching" (ICML 2025) — https://arxiv.org/abs/2502.05167 (peer-reviewed; effective length = length holding 85% of base score; most models below half by 32K once literal cues removed) ([litreview](llm_judge_litreview.md))