gwern's own page concludes the tank story did not happen, so citing it
undercut the exercise. Zech is peer-reviewed with the in-site against
out-of-site AUC pair; the fastbook case covers the tabular version.
Comment review mode only, no prose changed. Flags negative framing,
aphoristic closers, and three places where the rewrite made wassname's
hedged claims stronger than his original message.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Textbook order: collected advice, then his comment on how it applies to
LLMs, then the exercises. Content left for him to write.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Small is under a paragraph; large means work like comparing against a
reference repo. Do all applicable small ones, pick one large one.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Quotes come from the docs/evidence cache and were previously unused. Reuses the
existing footnote style, 14 new keys. Carries the source doc's coverage warning:
mode 6 has only three quotes and mode 7 has none that name similarity probes.
Source is his own message of 2026-08-25, spelling fixed and slightly more polite as he
asked, with each mistake pointing at the exercise that answers it. Also adds his rule
that a job is never abandoned without doing the exercises, one at a time.
Two gaps the existing 13 did not cover, found by mining the evidence cache against
wassname's list of common AI-agent failures. Quotes are verbatim from
docs/evidence/ (Steinhardt, Rahtz, Nanda, Goodfellow-Bengio-Courville).
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Interpretations must read as evidence for or against a claim, never as a cause,
because a cause list gets picked from, called certain, and used to stop.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Both were promised by the description and absent from the procedure. P2 now
prints the formatted examples; P3 opens with the bug assumption.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
A sonnet subagent had ml-debug listed, with 'after a run finishes or crashes' in
the description, and did not invoke it for 'read the last pueue log'.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Sonnet routed to P1 correctly then declined it: 'the task only asked to read, not
to audit or act'. Reading a log is exactly when the measurement table gets filled.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
A sonnet subagent read the skill, summarised the ritual list, ran none of them,
and closed with 'everything checks out'. Phase triggers do not fire; a sentence
the agent watches itself write might.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
ml-bench measured standalone answers, not whether an agent in a loop follows
principles. Those are different claims and the first does not support the second.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The run index is a cache key, not a seed, so subtracting run 3 from run 3 is
arbitrary. Unpaired the difference is +0.023 +- 0.044; by question +0.023 +- 0.031.
Still not pushed.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
+0.135 was one answer per arm, and it is draw 1 of three. Draws 2 and 3 read
-0.007 and -0.060, so the mean is +0.023 with sd 0.102. Not pushed: wassname
should read this before it goes public.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Quoted whole from the 2017 thread, since the article author asked to merge it
and never did. Verified line by line against the thread cache. Covers sample
size from a cumulative-mean plot, KLD/Dice on unbalanced data, augmentation
bounded by feature std, dummy metrics, jumpy validation loss, testing the
framework itself, activation swaps, and loss-curve shapes.
The author asked the commenter 'Do you mind if I add them to the article?' and
then did not: checked all 13 against the 2025 archived article, only the
batch-size point overlaps. Reddit blocks scrapers, so the thread came from a
Wayback snapshot. Also gave the article cache a real header: Medium is dead to
scrapers, so it now records the archive URL used to verify it.
The two README caches had no Source line, so nothing could re-verify them;
both now check at 99% coverage. The judge-bias file gets a warning at the top:
11 entries still carry summarizer numbers, and 2 of the 5 checked so far were
wrong.
102 of them, 48 already wrong after today's refetches. They rot every time a
cache is refetched and buy nothing the quote text does not: the caches are
verbatim, so the quote itself is the anchor. Descriptive labels stay.
All four came from the AI-generated deepwiki page, not the experiment log,
and one inverted the finding: the log says clipping was removed because
'Grad norm never exceeds 1.0 naturally', and the real distributed gotcha was
clipping local norms before sync. Footnote now points at dev/LOG.md on master
and at the renamed cache.
The notes file listed 11 links; 9 are already in the full page cache and the
80k Hours one has its own cache, so it only carried Rocktaschel, now in the
research_taste reading list. The Nanda shared draft was a 902-word excerpt of
a local download; wassname supplied the Google Doc, whose text export gives
the full 11318 words.
An /abs/ link is a stub: it costs a second lookup before anyone can check the
quote, and it is what let abstract-only caches look sourced. 77 links across
the skill files and 4 cache headers. Verbatim source bodies untouched, their
reference lists are the authors' text.
Refetches the URL in a cache file's header and measures 5-word shingle
coverage of the source, so a summary cannot pass as a copy. Guards against
stub sources, where high coverage of an abstract or landing page would
otherwise read as success. n=5 chosen by sweeping n against synthetic
copy/paraphrase/summary variants.
All nine said 'excerpted from HTML via browser' and held 141-684 words.
LessWrong posts refetched through the markdown API, the rest through jina.
All 99 previously quoted passages, and the 46 quotes SKILL.md and
refs/research_taste.md take from them, verify against the new full texts.
Every one was 3x to 100x shorter than its source: gwern_tank 179 -> 20132
words, cleanrl 37-details 239 -> 11410, kidger 185 -> 2932. Three passages the
old karpathy_recipe file presented inside quote marks were paraphrases, not
quotes, and are gone with the excerpts.
All were excerpt caches. Tuning playbook 662 -> 16216 words, ML Yearning
527 -> 26084 (the whole book, was ch13-19), bekman 579 -> 2357 (three
concatenated pages), Spinning Up 842 -> 3353. The Qwen3 report was truncated
mid-section 4.3 and now runs to the appendix.
Was 1179 words summarised from AI-generated deepwiki pages, accessed 2026-03.
Now the 9232-word primary dev/LOG.md, which also runs 2 months further.
Renamed since the source is the log, not deepwiki.
wassname