knob -> hyperparameter, gate -> threshold, rung -> defined level, clean log ->
error-free log, ship the rubric -> put the rubric. Quoted source text (Irpan's
canary, Domingos' knobs) and wassname's own lines are untouched.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
- refs/ -> references/, the folder name the Agent Skills spec uses and the one
Hermes skips when it walks for nested skills.
- rl and pinn declared name: rl and name: pinn, which are global names in a flat
skill namespace. Now ml-debug-rl and ml-debug-pinn. They also called themselves
sub-skills of 'ml-debugging', which is not this skill's name.
- Drop the dead link to SKILL_old.md. It moved into gitignored slop/, so the link
was broken for anyone who cloned.
- Route references/llm_judge_litreview.md, the one reference SKILL.md never named.
- Description leads with the trigger situations. Hermes truncates it to 57 chars.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Quotes come from the docs/evidence cache and were previously unused. Reuses the
existing footnote style, 14 new keys. Carries the source doc's coverage warning:
mode 6 has only three quotes and mode 7 has none that name similarity probes.
ml-bench measured standalone answers, not whether an agent in a loop follows
principles. Those are different claims and the first does not support the second.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
The run index is a cache key, not a seed, so subtracting run 3 from run 3 is
arbitrary. Unpaired the difference is +0.023 +- 0.044; by question +0.023 +- 0.031.
Still not pushed.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
+0.135 was one answer per arm, and it is draw 1 of three. Draws 2 and 3 read
-0.007 and -0.060, so the mean is +0.023 with sd 0.102. Not pushed: wassname
should read this before it goes public.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
SKILL.md is now folklore only: verbatim practitioner quotes ordered
most-general-first, transformer/LLM fine-tuning entries in their own
section, minimal context, links and footnotes. New sources: unsloth,
axolotl (+training stability), HF course ch8.4, Bekman debug_utils
(evidence frozen in docs/evidence/).
The synthesized material (mental models, priors, symptom tables, agent
loop, triage, anti-patterns) moves to PLAYBOOK.md, framed as menus of
hypotheses rather than authoritative diagnoses. Made-up symptom tables
no longer sit next to sourced quotes.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
NaN poisoning: inject NaN where info must not come from (future/test/labels), run the real pipeline, assert past outputs stay finite. Documents false negatives (pandas skipna, nanmean) and false positives (softmax rows, batch stats). Backprop-to-input is its gradient dual for inside the model; quote already frozen in docs/evidence/karpathy_recipe_training_nn_2019.md.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Reorder around what's durable, per wassname's curation:
- human-written intro up top; rename to "wassname's ML Debugging Folklore"
- mindset first: calibrate -> mental models -> Part 1 general tricks (kept,
they're well-based) -> read a working implementation when stuck
- a Folklore section built from verbatim, source-checked quotes (Jones,
Rahtz, Karpathy, Schulman, Henderson, Irpan, CS231n, Slavv, Goodfellow),
each footnoted to the canonical URL + the cached copy with line numbers
- LLM-agent babysitting (debugging loop, triage menu, anti-patterns) moved to
the bottom where it belongs; triage reframed as a menu, not a flowchart
- deeper one-off tricks split to refs/ (loss_surface, metric_stuck, sweeps),
scrubbed of private tooling (wandb/just/SI/personal scripts)
Quote integrity: every quote independently verified by fresh-eyes subagents
against the cached sources; fixed a reformatted Schulman slide, a truncated
Jones sentence, a reversed-order Rahtz stitch, a falsely-quoted Slavv phrase,
and the 3e-4 line (now the real tweet, framed as the joke Karpathy confirmed
it was, not gospel). lr_scheduler anti-pattern nuanced (warmup/cyclic matter).
Remove superseded SKILL2.md draft.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
Add 3 new evidence files from modern open-source sources:
- karpathy_recipe_training_nn_2019.md: Karpathy's training recipe blog post
- nanochat_deepwiki_llm_pretraining_2026.md: 320+ HP sweeps for GPT-2-scale pretraining
- sanh_simple_considerations_hf_2021.md: HuggingFace NLP debugging notes
Add update-to-data ratio diagnostic to refs/diagnostics.md (target ~1e-3).
Add LLM pretraining gap note to SKILL.md intro linking the new sources.
Add tanh saturation % to logging checklist.
wassname