From 6c504961229bc61b7b61e16d7d615c6776f99bd6 Mon Sep 17 00:00:00 2001 From: wassname <1103714+wassname@users.noreply.github.com> Date: Sat, 15 Aug 2026 06:23:54 +0800 Subject: [PATCH] add missing cache headers, warn on the unverified judge entries The two README caches had no Source line, so nothing could re-verify them; both now check at 99% coverage. The judge-bias file gets a warning at the top: 11 entries still carry summarizer numbers, and 2 of the 5 checked so far were wrong. --- docs/evidence/llm_judge_biases.md | 9 +++++++++ docs/evidence/reports/code/nous_atropos_readme.md | 4 ++++ docs/evidence/reports/code/olmo_core_readme.md | 4 ++++ 3 files changed, 17 insertions(+) diff --git a/docs/evidence/llm_judge_biases.md b/docs/evidence/llm_judge_biases.md index a91ef7d..425527a 100644 --- a/docs/evidence/llm_judge_biases.md +++ b/docs/evidence/llm_judge_biases.md @@ -5,6 +5,15 @@ Fetch-status: verbatim (abstracts in full or near-full; verdict pages are short # LLM judge biases (excerpts) +WARNING, read before quoting anything from this file. It was assembled by +agents across several sessions and the provenance is uneven. Entries tagged +[FT] were copied from raw source. Entries tagged [ID] have numbers a +summarizer produced from a web page, and nobody has read the paper. On +2026-08-15 I re-pulled the five [ID] entries the litreview depends on and two +of the five carried a wrong number, so treat the remaining 11 as roughly 2-in-5 +wrong until each is checked against raw text. Do not promote an [ID] number +into SKILL.md or refs/ without re-pulling the paper first. + ## "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" — Zheng et al. (LMSYS), NeurIPS 2023 — https://arxiv.org/pdf/2306.05685 The canonical paper naming the bias taxonomy: diff --git a/docs/evidence/reports/code/nous_atropos_readme.md b/docs/evidence/reports/code/nous_atropos_readme.md index 4d12f59..3d26b1b 100644 --- a/docs/evidence/reports/code/nous_atropos_readme.md +++ b/docs/evidence/reports/code/nous_atropos_readme.md @@ -1,3 +1,7 @@ +Source: https://github.com/NousResearch/atropos/blob/main/README.md +Title: Atropos, Nous Research's LLM RL Gym (repository README) +Fetch-status: verbatim README + # Atropos - Nous Research's LLM RL Gym ![newatr-02](banner-image.jpg) diff --git a/docs/evidence/reports/code/olmo_core_readme.md b/docs/evidence/reports/code/olmo_core_readme.md index 492a597..6d5ec94 100644 --- a/docs/evidence/reports/code/olmo_core_readme.md +++ b/docs/evidence/reports/code/olmo_core_readme.md @@ -1,3 +1,7 @@ +Source: https://github.com/allenai/OLMo-core/blob/main/README.md +Title: OLMo-core, building blocks for OLMo modeling and training (repository README) +Fetch-status: verbatim README +
OLMo Logo