Add randomized ML debugging fortunes

Co-Authored-By: PI[k3] <288921227+claudypoo@users.noreply.github.com>

README.md

@@ -23,6 +23,58 @@ Or paste `SKILL.md` into your system prompt / context when debugging.
 ## Folklore
 
 
+### The rules, before the rules (Agans)
+
+Most of this folklore's lineage goes back to a 2002 debugging book for general
+electronics and software. Its nine rules, in full, from chapter 2:[^agans]
+
+> UNDERSTAND THE SYSTEM
+> MAKE IT FAIL
+> QUIT THINKING AND LOOK
+> DIVIDE AND CONQUER
+> CHANGE ONE THING AT A TIME
+> KEEP AN AUDIT TRAIL
+> CHECK THE PLUG
+> GET A FRESH VIEW
+> IF YOU DIDN'T FIX IT, IT AIN'T FIXED
+
+Each rule is worth the full Remember summary at the end of its chapter. The
+ones that map most directly onto agent debugging:
+
+> **Quit Thinking and Look**: You can think up thousands of possible reasons
+> for a failure. You can see only the actual cause.
+>
+> See the failure. The senior engineer saw the real failure and was able to find the cause. The junior guys thought they knew what the failure was and fixed something that wasn't broken.
+> See the details. Don't stop when you hear the pump. Go down to the basement and find out which pump.
+> Build instrumentation in. Use source code debuggers, debug logs, status messages, flashing lights, and rotten egg odors.
+> Add instrumentation on. Use analyzers, scopes, meters, metal detectors, electrocardiography machines, and soap bubbles.
+> Don't be afraid to dive in. So it's production software. It's broken, and you'll have to open it up to fix it.
+> Watch out for Heisenberg. Don't let your instruments overwhelm your system.
+> Guess only to focus the search. Go ahead and guess that the memory timing is bad, but look at it before you build a timing fixer.
+
+> **Change One Thing at a Time**: You need some predictability in your life.
+> Remove the changes that didn't do what you expected. They probably did
+> something you didn't expect.
+>
+> Isolate the key factor. Don't change the watering schedule if you're looking for the effect of the sunlight.
+> Grab the brass bar with both hands. If you try to fix the nuke without knowing what's wrong first, you may have an underwater Chernobyl on your hands.
+> Change one test at a time. I knew my VGA capture phase was broken because nothing else was changing.
+> Compare it with a good one. If the bad ones all have something that the good ones don't, you're onto the problem.
+> Determine what you changed since the last time it worked. My friend had changed the cartridge on the turntable, so that was a good place to start.
+
+> **If You Didn't Fix It, It Ain't Fixed**: And now that you have all these
+> techniques, there's no excuse for leaving it unfixed.
+>
+> Check that it's really fixed. Don't assume that it was the wires and send that dirty fuel filter back onto the road.
+> Check that it's really your fix that fixed it. "Wubba!" might not be the thing that did the trick.
+> Know that it never just goes away by itself. Make it come back by using the original Make It Fail methods. If you have to ship it, ship it with a trap to catch it when it happens in the field.
+> Fix the cause. Tear out the useless eight-track deck before you burn out another transformer.
+> Fix the process. Don't settle for just cleaning up the oil. Fix the way you design machines.
+
+Full verbatim chapter summaries are in the [evidence notes](docs/evidence/agans_debugging_9_rules.md);
+the complete book text lives in the dlbook repo.
+
+
 ### Think more, experiment less
 
 > before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possiblity and brainstorm the cheapest tests that may narrow them down. - wassname
@@ -617,6 +669,7 @@ Folklore sources (the quotes above trace to these):
 [^sanh]: Victor Sanh, "Simple considerations for simple people building fancy neural networks" (HF, 2021) -- https://huggingface.co/blog/simple-considerations ([cache](docs/evidence/sanh_simple_considerations_hf_2021.md): decent-performance-without-crashing, read-the-tokenizer-output, 4e2-is-a-symptom, pre-training questions)
 [^steinhardt]: Jacob Steinhardt, "Research as a Stochastic Decision Process" -- https://cs.stanford.edu/~jsteinhardt/ResearchasaStochasticDecisionProcess.html ([cache](docs/evidence/steinhardt_research_stochastic_decision_process.md): 0.1%-of-implementations, high-standard-for-ruling-out, months-of-approaches-one-cause)
 [^miller]: Evan Miller (Anthropic), "Adding Error Bars to Evals" (2024) -- https://arxiv.org/pdf/2411.00640 ([cache](docs/evidence/miller_2024_error_bars_evals.md): five recommendations, question-level pairing, power analysis). arXiv preprint, not peer reviewed.
+[^agans]: David J. Agans, *Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems*, AMACOM, 2002 ([notes](docs/evidence/agans_debugging_9_rules.md): nine rules and Remember summaries verbatim; complete book text in the private dlbook repo)
 [^fsdl]: Josh Tobin, Full Stack Deep Learning Spring 2021 lecture 7, "Troubleshooting Deep Neural Networks", notes by James Le and Vishnu Rachakonda -- https://fullstackdeeplearning.com/spring2021/lecture-7/ ([cache](docs/evidence/fsdl_spring2021_lecture7.md): error up/explodes/oscillates/plateaus table)
 [^olsson]: Catherine Olsson and the 80,000 Hours team, "ML Engineering for AI Safety and Robustness" -- https://80000hours.org/articles/ml-engineering-career-transition-guide/ ([cache](docs/evidence/olsson_80000hours_ml_engineering_ai_safety.md): bug-hunting-with-diagnostics-over-tuning). Reports Daniel Ziegler's self-study second-hand.
 [^reddit-rl]: u/GrundleMoof, "How to more intelligently debug RL roadblocks?" -- https://old.reddit.com/r/reinforcementlearning/comments/bzg3l2/ ([cache](docs/evidence/reddit_rl_roadblocks_bzg3l2.md): nine-knobs list, terminal-flag and broadcast bugs in the replies). Anonymous self-report from a self-described non-expert; quoted as a specimen of the failure mode, not as authority.

SKILL.md

@@ -1,226 +1,308 @@
 ---
 name: ml-debug
-description: "Debug an ML run: read the log, it crashed, the loss will not go down, the metric will not move, is this result real, does A beat B, a spike or anything weird in the log, about to queue a run, or about to write that a result looks fine. Machine learning debugging exercises, each under a quote from a practitioner. Do the exercise for your situation and show the result in your reply. Invoke it yourself."
+description: "Debug an ML run: read the log, it crashed, the loss will not go down, the metric will not move, is this result real, does A beat B, a spike or anything weird in the log, about to queue a run, or about to write that a result looks fine. Fill the ml-debug form and do the exercises that match your situation. Show the results in your reply. Invoke it yourself; deciding a run does not need it is the behaviour being tested."
 ---
 
-In an attempt to upskill the machine learning debugging on AI coding assistants (and humans), I've collected high quality sources on how to debug machine learning projects, focusing on the mindset and the "taste". When I started ML I went searching for discussions on best practices, and started a few discussions of my own and they helped me a lot, over the years I've collected good ones. I hope they can help others, as well as help in auto research setups. This intro is human written, and the below is AI written with human guidance. - wassname
+Sources, the human-written introduction, and frozen copies of every quote are in
+[README.md](README.md). Paragraphs signed "- wassname" are his. Paragraphs with a `CLAUDE:`
+comment are Claude's wording, with the source of the point stated.
 
 Be diligent. Work the problem in full before you write. State the decisive point early, then give
 the derivation, the mechanism, or the log line behind it, so the reader can check it and not just
 take it. Show the work, not only the conclusion.
-<!-- CLAUDE: top of the file because it is the one part with measured uplift. See the results
-table in README.md. Do not bury it. -->
+<!-- CLAUDE: at the top because it is the one part with measured uplift; see README results. -->
 
-## Task: The ml-debug form
+## How ML debugging differs
 
-Your task is to fill in the below form and show it in full to the user.
+> broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task. -- Achiam
 
-To do this, read the full log, and think about the architecture, training dynamics, bugs, etc and
-show the user you have thought about it. Think step by step, be diligent, avoid overconfident or
-narrow perspectives, in order to complete this critical thinking exercise. You will be evaluated on
-the form. Write "unknown" in a row you cannot fill, and say what would fill it.
-
-- 3+ hypothesis or diagnosis with % bet on each
-- Q: Strange or unexpected observations from the log, quote the lines and explain why they are
-  surprising
-  - TODO Have blind subagent read log and fill out this form, quote it's answer too
-- Is it:
-  - Q: Overfitting, underfitting, **loss** blowing up.
-  - You should use OneCycleLR, if so, at what **lr** does it start and stop learning (this can tell
-    you ceiling and floor)?
-  - Q: Is the **init** giving 0 change, or too much (incoherent / exploding change), and does it
-    beat baseline?
-    - Note that it should work at init, or else your data or architecture has a bug
-  - Q: Does it beat **dummy** (persistance, null model, or simple heuristic) at each stage?
-    - Note it should do this easily
-  - Q: Does it beat **baseline models** at each stage?
-    - Note you are not finished untill you do this in val and held out
-  - Q: How long does each stage take, and how much GPU ram? Can we **speed up** the iteration cycle?
-  - Q: Have you viewed one inference including **sample data**, model output, and reasoning to
-    ensure it aligns with expectations? You should print one sample of input, output, and everything
-    else each time to sanity check and bring sunlight. Q: Link or quote one full sample or plot of
-    each type
-  - Advanced: TODO There is always a bug: find the most likely one, and have a blind subagent find
-    the most likely one and quote both here to user
-
-The task is complete when you have shown the filled form AND the routing list below, with every
-branch whose condition is true. How it is scored:
-
-- a row answered from memory or from what you expected, rather than from a quoted log line: 0
-- a row left blank without "unknown" and what would fill it: 0
-- deciding this run does not need ml-debug and stopping here: 0. That decision is the behaviour
-  being tested, not an exemption from the test.
-<!-- CLAUDE: wassname's point, my wording. He said "the idea is to stop it being lazy, force it to
-go through the routing and think. many agents are trained to skim, and some are trained to sandbag
-and avoid ml-debug", and to frame it as a task close to the training env. -->
-
-
-> It's normal to want to rush into training and evaluating models, but it's important to take the time to think about the goals of a project, to fully understand the data that will be used to support these goals, to consider any limitations of the data that need to be addressed, and to understand what's already been done in your field. -- Lones
-
-> This *sounds* obvious, but in practice this requires constant active effort, and if you are not actively doing this you'll inevitably fall into traps. Always seek alternative explanations, seek and implement strong baselines, check for bugs, etc. -- Nanda
-
-Then do the exercises for your situation and show the result in your reply. Each
-exercise is marked (small) or (large). A small one takes less than a paragraph to answer; a large
-one is a lot of work, such as searching for a reference repo and comparing against it.
-
-Always do ex #1 and ex #3. Then walk the list and do every branch whose condition is true. Do all
-the small ones you match, and one large one.
-
-- always, whatever you are doing
-  - ex #1 read the log end to end (small)
-  - ex #3 read your data (small)
-- before a run
-  - if about to queue it: ex #5 list the options you have (small), ex #6 write down what you
-    expect to see (small)
-  - if about to change the design, or you cannot explain the last run: ex #13 pseudocode and
-    external review (large)
-- after a run
-  - if it finished or crashed: ex #2 name a second cause for the same number (small), ex #4 chase
-    the weird thing (small)
-  - if the log looks weird, a spike or a flat line or an impossible value: ex #11 read the rows
-    before the spike (small), then ex #10 localise the error (small)
-- before you report
-  - if about to set a pass threshold: ex #15 get the scale before the threshold (large)
-  - if about to quote a headline metric: ex #12 name what else could score well (small)
-  - if about to say you found the cause: ex #7 multiple diagnoses with % bets (large)
-  - if about to say A beats B: ex #8 three ways the result is false (large)
-  - if about to call it negative: ex #14 one implementation is not the idea (small)
-- if two cycles have passed with no progress
-  - ex #9 compare against a reference implementation (large)
-
-Each exercise says what to show. Show it in full: the table, the quoted log line, the quoted
-code, the pasted sample. Write "unknown" in a cell you cannot fill, and say what would fill it.
-Give the source of each number.
-
-Never stop a job or give up on an idea without doing all of these. One at a time, not all at once.
-
-> **NEVER STOP**: Once the experiment loop has begun (after the initial setup), do NOT pause to ask the human if you should continue. Do NOT ask 'should I keep going?' or 'is this a good stopping point?'. The human might be asleep, or gone from a computer and expects you to continue working *indefinitely* until you are manually stopped. You are autonomous. If you run out of ideas, think harder — read papers referenced in the code, re-read the in-scope files for new angles, try combining previous near-misses, try more radical architectural changes. The loop runs until the human interrupts you, period. -- Karpathy, [autoresearch/program.md](https://github.com/karpathy/autoresearch/blob/master/program.md)
-<!-- annoy-less: [#9 negative framing + clipped fragment] "One at a time, not all at once." is an
-X-not-Y closer in the AI register. Written by CLAUDE, your call. -->
-
-
-## Common mistakes
-
-Everyone makes these, and I have made most of them myself. They come up so often with AI agents in
-long autoresearch runs that they are worth naming, so you can catch yourself early rather than after
-a week of work. Reading the log and hunting for your own bug are the two that do most of the damage,
-so start there when you are not sure where to start. - wassname
-<!-- annoy-less: [MOST IMPORTANT - confidence changed] this whole paragraph is signed wassname but
-was written by CLAUDE from your chat message. Your original last clause was:
-"reading the log and looking for bugs are the most comomn I guess"
-CLAUDE dropped "I guess" and changed "most common" to "do most of the damage", which is a different
-and stronger claim. Restore your hedge if you want your own confidence level back. -->
-<!-- annoy-less: [invented detail] "I have made most of them myself" and "rather than after a week
-of work" are CLAUDE's, not from your message. First person claims about you that you did not make. -->
-
-
-> Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research. -- Nanda
+> If one part is broken, the other parts can adapt and still achieve roughly acceptable performance -- Goodfellow, Bengio and Courville
 
 > The challenge lies in the fact that you can make these mistakes, train a model without it ever crashing, and still get a decent performance... -- Sanh
 
-Be careful about being overconfident. It is easy to write a diagnosis in the tone of a fact. Before
-you commit to one, ask what you saw that a competing explanation could not also explain. If nothing,
-then "I do not know, and here is what would tell me" is a good answer and not a failure.
-Ex #7 multiple diagnoses with % bets.
-<!-- annoy-less: [#9 negative framing] "is a good answer and not a failure" is the X-and-not-Y
-closer. Say the positive claim only. Written by CLAUDE. -->
+The training script has to print the checks, as SHOULD lines written before the run and
+compared after it.
+<!-- CLAUDE: one line from the three quotes above. -->
 
+### Expensive runs
 
-Do not quit after the first change and call the negative real. One failed attempt is much more
-likely to be a bug in your implementation than a refutation of the idea. This is the expensive
-mistake, because the idea gets thrown away and nobody goes back to it. Look for the bug first.
-Ex #14 one implementation is not the idea.
-<!-- annoy-less: [confidence changed + significance narration] your original was "quit after the
-first change to misdiagnose a negative". CLAUDE added "much more likely to be a bug" (a probability
-you did not state) and "This is the expensive mistake", which tells the reader how to rate it. -->
+> Although one might think we would spend most of our time trying to maximize performance on the validation set, in practice we spend the majority of our time trying to gain insight into the problem -- Godbole, Dahl, Gilmer, Shallue and Nado
 
+If it takes 5 hours to run, we might only get 4 runs a day, so we need to make them as
+informative as possible. We can't schedule a sweep or ablation of 100+ runs, so we make multiple
+changes that will have separate and distinguishable effects on the metrics. What you learn is the
+effect of each change given the others, so record it that way in the mental model. - wassname
+<!-- CLAUDE: last sentence is mine (Sculley's CACE, in README). -->
 
-Try not to stop at the first idea you come up with. It arrives with no competition, so it wins by
-default rather than on merit. Write down two more, and say what observation would separate them. If
-you cannot name a test that distinguishes them, you have a preference and not a hypothesis.
-Ex #6 write down what you expect to see, ex #7 multiple diagnoses with % bets.
-<!-- annoy-less: [#9 negative framing] "you have a preference and not a hypothesis" is again the
-X-and-not-Y closer. Two of these in one section reads as a formula. Written by CLAUDE. -->
-<!-- annoy-less: [aphorism] "It arrives with no competition, so it wins by default rather than on
-merit." is CLAUDE's epigram, not in your message. Cut or say it plainly. -->
+### How agents fail
 
+> Trying an experiment and seeing it fail gives little information by itself. When an experiment fails, it is tempting to conclude "I tried X and it didn't work". However, if X is a high-level conceptual approach, then a more correct conclusion is "I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work". -- Steinhardt
 
-> If it doesn't work, assume there's a bug. Spend a lot of effort searching for bugs before you resort to tweaking hyperparameters: usually it's a bug. Bad hyperparameters can significantly degrade RL performance, but if you're using hyperparameters similar to the ones in papers and standard implementations, those will probably not be the issue. -- Achiam
+> Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research. -- Nanda
 
-Watch out for getting obsessed with the legible hyperparameters. Learning rate, batch size and
-warmup are easy to name and easy to change, so they attract more attention than they deserve. More
-often the cause is in the data, a sign, a mask, an index, or a metric that answers a different
-question from the one you asked. Ex #5 list the options you have, ex #10 localise the error.
-<!-- annoy-less: [confidence changed] "More often the cause is in the data" is a frequency claim
-CLAUDE added; your message only listed the obsession, not a base rate. -->
+Nanda's "fail fast" advice is for a human who over-commits to a direction for a year. Agents fail
+the other way: they skim the log until a line looks like a reason to stop, find a reading of the
+task that permits stopping, or change one hyperparameter and call the idea dead. The other habits
+this file is written against: settling on the first hypothesis because it arrived first; treating
+learning rate and batch size as the whole option space; writing a probe script beside the
+training script, which then has its own bugs; reading the last twenty lines of the log; and
+writing a diagnosis in the tone of a fact when a competing explanation fits the same evidence.
+<!-- CLAUDE: wassname's observations from autoresearch runs ("give up too easy", "skim until they
+find a reason", side-cars, hyperparameter obsession); my wording. -->
 
+## What to keep in the repo
 
-Please read the data. Print the first full training sample, chosen and rejected, with the special
-tokens and the loss mask showing. Look at it with your own eyes. Most formatting bugs are obvious in
-the first sample and invisible in every aggregate. Ex #3 read your data.
-<!-- annoy-less: [antithesis formula] "obvious in the first sample and invisible in every aggregate"
-is a balanced-opposites flourish. CLAUDE's phrasing, and "Most" is an added frequency claim. -->
-
-
-Please read the log. Not the last twenty lines, the log. Find the first line where the run stopped
-matching what you expected, quote it, and start from there. Ex #1 read the log end to end,
-ex #11 read the rows before the spike.
-<!-- annoy-less: [#9 negative framing, clipped fragment] "Not the last twenty lines, the log." is a
-sentence fragment in the not-X-but-Y shape. It may still be the clearest way to say it, your call.
-Written by CLAUDE. -->
-
+Defaults for a long research loop (runs of an hour or more, a novel method, an agent working
+overnight). A short debugging call on an existing script creates none of these.
 
 Do not write a side-car probe script. Build up the one training script so it has all the metrics
 you need inline as you go, with short interpretable demos at many stages: init, mid train, post
 train, eval, then one long unclipped demo at the end. Demos and probes should not be separate
 runs, they should be quick sanity checks inside the main train script, and the script should write
-`log.md` in markdown (see `token-efficient-logging` and `markdown-tables`) so the log diagnoses in
-situ instead of needing a second pass. That is how a lot of nights get wasted and agents go off
-track: they make side-cars with their own separate bugs and weird correlational measurements, and
-have nothing to show for it. If we work on the training script we watch it get better, we reuse
-the same code, we understand it better, and we squash the bugs. - wassname
+`run.md` in Markdown so the log diagnoses in situ instead of needing a second pass. That is how a
+lot of nights get wasted and agents go off track: they make side-cars with their own separate bugs
+and weird correlational measurements, and have nothing to show for it. If we work on the training
+script we watch it get better, we reuse the same code, we understand it better, and we squash the
+bugs. - wassname
 
-A cosine probe is the usual side-car, and `cos(apple, orange) = 0` is not a null result. Ex #2.
+`train.py`. One file. The novel part is written as a readable narrative with tensor shapes in
+comments, so a reviewer can follow it top to bottom without opening other files.
 
+Each long run owns `outputs/<date>_<slug>_<seed>/`: resolved config, commit and argv provenance,
+`run.md`, rectangular metrics, ragged demos/generations, and checkpoints. A detached reader must
+be able to reconstruct and sanity-check the run from that directory.
 
-> *   How would a random predictor perform (especially in classification problems)? Dataset can be unbalanced...
-> *   What would the loss look like for a random predictor?
-> *   What are the limits of this metric? If it's perfect, what can I conclude? What can't I conclude? -- Sanh
+`run.md`, written by the training entry point, is valid Markdown and the result page. Start each
+stage with a heading and breadcrumb, then close it with elapsed time and peak GPU memory when
+relevant. Include the resolved config actually used; a decimated (about 30 to 60 row) metrics table;
+the first train and evaluation examples in raw form and as the model consumes them (for a
+transformer, special tokens and loss mask visible); and one full normal-path demo for every
+LLM-facing stage that exists. Keep stdout sparse and print the log path. Re-emit a compact final
+result block: headline metric, full copyable result table, output path, and run identity.
 
-Do not fix on an arbitrary metric threshold before you have any idea what a fair or good threshold
-is. Saying the metric must clear 0.8 means nothing until you know what counts as good here. Get the
-scale first, from a null arm and a shuffled control. Ex #15 get the scale before the threshold.
-<!-- annoy-less: [invented example] "Saying the metric must clear 0.8 means nothing" - the 0.8 is
-CLAUDE's, not from your message. Fine as illustration, but it is not your number. -->
+Keep `TODO validate:`, `FIXME:`, or `SHOULD:` beside the evidence it interprets. `SHOULD:` needs a
+mechanism, derivation, paper, or validated prior run; otherwise use `TODO validate:`. It carries a
+number only after the scale exercise (ex H) has been done.
 
+For a comparative result table: first column is an index linked to source, then short metadata,
+then the headline score and its inputs. Sort by the headline score; put an arrow on every header;
+bold meaningful per-column best cells; italicize controls and baselines; include floors; and use
+one table for each comparable group. Put the headline result and output path at the end of `run.md`.
 
-> `try/except` around training code. Training should crash loudly. A caught exception hides the bug and produces silently wrong results. The one exception is checkpoint-on-KeyboardInterrupt. -- from [PLAYBOOK.md](PLAYBOOK.md)
+The raw event trace is the source of truth. Keep JSONL or Inspect records verbatim and link from
+`run.md` with a project-relative path and line where possible. Do not summarize away a failed,
+truncated, incoherent, refusing, saturated, or confounded output.
 
-Do not write code that carries on after it has already failed. A load that loaded nothing, a filter
-that matched nothing, a config key that was missing, all of these should stop the run rather than
-hand you an error-free log and a wrong result. Assert that the thing you asked for is there. The cost of
-this one is measured in runs, not minutes: a `strict=False` that quietly loaded no weights hid a
-dead experiment arm for eight runs in my own repo. Ex #2 name a second cause for the same number,
-ex #7 multiple diagnoses with % bets.
+A smoke test before every costly run: execute the real pipeline end to end on a tiny random model
+and small slice of every train, extract, and evaluation stage. Use real loaders, I/O, LLM calls,
+and evaluation; reduce scale only. Annotate function inputs and outputs with `jaxtyping`, and
+activate `beartype` only for this smoke run (for example, `BEARTYPE=1`). Garbage scores are fine:
+it checks code paths, shapes, and dtypes, not scientific validity. A flipped sign, label leakage,
+an all-`-100` mask, or a bad metric can pass it.
+<!-- CLAUDE: direct compact integration of token-efficient-logging, markdown-tables, setup-repo,
+and jaxtyping. -->
 
-A separate thing that shares the name "fail fast", and worth keeping separate in your head:
+`MENTAL_MODEL.md`, under two pages. What you believe about this system: which changes
+(regularisation, architecture, a bottleneck, loss balance, more data, init scale, optimiser)
+move which metrics, in which direction, and with what credence. Updated after every run in a
+Bayesian way: a credence moves on a cited log line, and a disproved row is marked disproved with
+the line rather than deleted. Read it at the start of every turn. The filled form for each run is
+appended to whatever run log the repo already keeps.
+<!-- CLAUDE: wassname asked for one file; this is his description of its contents. Experimental,
+he has not worked with it yet. -->
 
-> **Fail fast**. One of the largest time sinks possible is **investing weeks to months of effort into a failed research direction**. [...] It's often much better to have several quick and dirty experiments to attack different angles where you could fail fast than to put a lot of effort into one. -- Nanda
+## The ml-debug form
 
-That one is about killing a doomed direction early. The one above is about crashing on the error.
-Both are good and they are not the same rule.
+Fill this in and show it in full. Read the whole log first. Scoring:
 
-Read the first one with its audience in mind. Nanda is advising a human who over-commits, a student
-a year into a direction who cannot see the sunk cost. Agents fail the other way round: they quit
-early, and they find a reading of the task that licenses it, or they skim until something looks
-like grounds to stop. So the rule does not transfer unchanged. Before you call a direction dead,
-do ex #7 and ex #9 and show the result: what you expected, what you got, and the bug you ruled
-out. A reason found while skimming does not count.
-<!-- CLAUDE: wassname's point, my wording. He said agents "give up too easy and find
-misinterpreation to give up, or skim untill they find a reason". -->
+- a row answered from memory or expectation, with no quoted log line: 0
+- a row left blank, with no "unknown" and no note on what would fill it: 0
+- deciding this run does not need the form: 0. That decision is the behaviour being tested.
 
+> Read your data. Often, the quality of the data is a crucial driver of the results of your experiments. Often, it is quite bad. -- Nanda
 
-## How this applies to LLM agents
+> How would a random predictor perform (especially in classification problems)? [...] What would the loss look like for a random predictor? [...] What are the limits of this metric? If it's perfect, what can I conclude? What can't I conclude? -- Sanh
+
+| row | answer |
+|---|---|
+| log length; the config as it appears in the log | |
+| each `SHOULD:` line, then the observed line, quoted | |
+| for every number you cite: its value under a null (chance, ln C, the base model, a random predictor) and where that expectation came from | |
+| at init, before any update: what did the demo show, and how does it compare to the base model or to chance? | |
+| against a dummy (persistence, class prior, null model, simple heuristic) at each stage: which wins, by how much? | |
+| against the baseline model at each stage, on val and on held-out: which wins? | |
+| if the schedule ramps (warmup, OneCycle): at what lr did learning start, at what lr did it stop? | |
+| one full sample, viewed: input as consumed, output, trace. Link or quote it | |
+| at the worst-looking step: loss per term, grad norm per module. Which module does it point to? | |
+| lines in the log that surprised you, quoted, with why. Each ends "explained: ..." or "chasing now" | |
+| what is not in this log that you would need in order to trust it | |
+| three or more diagnoses with a % on each: one bug in the training code, one bug in the eval, one confound or shortcut, some % on unknown. For each, the strongest evidence for and against, from the log. No evidence against means untested | |
+| a fresh subagent, given the training entry point and `run.md` with no diagnosis attached, asked for the top bugs and misconceptions. Its list, quoted, including "found nothing" | |
+| the cheapest test separating the top two diagnoses, and what each predicts | |
+| wall-clock and GPU memory per stage; what would shorten the loop | |
+
+Some rows are an exercise below at less depth. The form is done every time; the exercise is done
+at depth when the routing says so.
+
+## Routing
+
+Before a run, after a run, before you report. At each, do every small item that applies and one
+large item. A small item is under a paragraph. A large one is real work.
+
+Before a run:
+- always: options table (ex A, small), predictions (ex B, small), smoke test
+- if about to change the design, or the last run cannot be explained: pseudocode and external
+  review (ex F, small; the review is delegated)
+
+After a run (finished or crashed):
+- always: the form; second cause for the same number (ex C, small)
+- if it failed: reproduce it, same seed then a different seed, before diagnosing. A failure
+  that does not reproduce is a different problem; write that down
+- if the log has a spike, a flat line, or an impossible value: rows before the spike (ex D, small)
+- if two cycles have passed with no progress: reference implementation (ex E, large)
+
+Before you report:
+- if about to quote a headline metric: what else could score well (ex G, small)
+- if about to set a threshold: the scale first (ex H, large)
+- if about to say A beats B: three ways it is false (ex I, large)
+- if about to call it negative: one implementation is not the idea (ex J, small), then ex I on
+  your own code
+
+After a change to `train.py` improves a metric: quote the line that moved and give the mechanism
+by which the change moved it. Agans' ninth rule, "if you didn't fix it, it ain't fixed": an
+improvement you cannot explain means something else is compensating.
+<!-- CLAUDE: Agans (docs/evidence/agans_debugging_9_rules.md); the compensation reading is mine,
+via Goodfellow's "other parts can adapt" above. -->
+
+Reference search (ex E), external review (ex F), and the blind reads in the form and ex I are
+subagent jobs, for the same reason each time: the subagent has no diagnosis to defend. The
+diagnosis stays in the main context.
+<!-- CLAUDE: wassname's point that exploring, searching and reviewing suit subagents. -->
+
+In an autoresearch loop, where the human has left and expects the loop to keep running:
+
+> **NEVER STOP**: Once the experiment loop has begun (after the initial setup), do NOT pause to ask the human if you should continue. Do NOT ask 'should I keep going?' or 'is this a good stopping point?'. The human might be asleep, or gone from a computer and expects you to continue working *indefinitely* until you are manually stopped. You are autonomous. If you run out of ideas, think harder — read papers referenced in the code, re-read the in-scope files for new angles, try combining previous near-misses, try more radical architectural changes. The loop runs until the human interrupts you, period. -- Karpathy, [autoresearch/program.md](https://github.com/karpathy/autoresearch/blob/master/program.md)
+
+A job is stopped, or an idea dropped, only after the form, ex I, and ex J are written out.
+
+## Exercises
+
+### ex A: options table (small)
+
+> Build it up as you go, don't think you can build it ahead of time. Be focused on a strong mental model of what options you have (including architectural changes and losses) that you think should affect what metrics in the logs. -- wassname
+
+The table lives in `MENTAL_MODEL.md` (or in your reply, for a short call). Correct it before each
+run and show it.
+
+| option | metric it should affect | direction and order | what separates it from the other options |
+|---|---|---|---|
+
+Consider architecture and loss changes where they are live choices for this problem, alongside
+data, regularisation, and optimiser. Say which options change in this run and why. Several can
+change in one run if each has its own metric (see Expensive runs). Show the config diff against
+the run you will compare to.
+
+### ex B: predictions (small)
+
+> Before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possibility and brainstorm the cheapest tests that may narrow them down. -- wassname
+
+Write down the question this run answers in one sentence, the result that would make you drop the
+idea, and which part is the novel part (everything else is a control). Then:
+
+| risky part | what I expect to see | too weak | too strong | buggy | metric exists? |
+|---|---|---|---|---|---|
+
+Add to `train.py` every metric whose last column says no. The controls: the base model on the same
+inputs; a random direction or shuffled labels through the same pipeline; the method with the novel
+part removed; the metric on data not used to build the intervention. Say how many seeds. Queue the
+run so its finish wakes you, and use the wait to sharpen the predictions.
+
+### ex C: second cause for the same number (small)
+
+> What I'm advocating for here is not a blind faith in the buginess of your code, but for dramatically raising the threshold at which you start thinking 'OK, I think this is correct.' -- Jones
+
+Which number does the diagnosis rest on? Quote the code that computes it. What else would produce
+that number, and what second metric separates the two? A cosine near 1 can be a shared mean or a
+collapsed latent. A cosine of 0 between two probe directions says they are orthogonal and nothing
+about whether either probe works, so it rules nothing out.
+
+### ex D: rows before the spike (small)
+
+> As you can see it's the previous frames that we need to look into when the numbers start going into very large for fp16 numbers. -- Bekman
+
+For each spike or collapse, show the rows before it and say which column moved first.
+
+### ex E: reference implementation (large; subagent)
+
+> We find that implementation differences which are often not reflected in publications can have dramatic impacts on performance. -- Henderson
+
+> If you are stuck, find a working reference implementation and compare it to yours. If nothing jumps out, try a bisection search: adapt their code wholesale, then half their features, and so on. -- wassname
+
+Search for implementations of the nearest method. Rank by: a results table, an issue or note
+saying someone else reproduced it, more than one human contributor, a README with evaluation
+details, other repos that import it. Take the top one or write "no reference exists".
+
+| feature | theirs (file:line) | mine | same? |
+|---|---|---|---|
+
+Include algorithm tweaks, engineering tricks, hyperparameters, and logged metrics. Ask the
+subagent for at least one bug in your module.
+
+### ex F: pseudocode and external review (small; review delegated)
+
+> Summarise your concept and pseudocode and do an external review in scientist mode. Perhaps describe the forward and backward pass as mermaid too. -- wassname
+
+Write the concept in plain English, then compact Python-shaped pseudocode: use Unicode math names
+when they match the method, `←` for conceptual assignment, shapes in trailing comments, and
+parameter counts per module. Omit imports, device moves, error handling, and other boilerplate.
+Add a Mermaid forward/backward diagram when it clarifies the design. Give this material, and no
+diagnosis, to a fresh reviewer from a different model family where one is available. Ask for its
+assumptions, likely bugs, and first test. Show its verdict; if no reviewer is available, say so in
+the report.
+
+### ex G: what else could score well (small)
+
+> The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al.
+
+> Apparently meaningless identifier columns were the most important predictors. [...] the university only filled out much of this information *after* a grant application was accepted. -- Howard and Gugger
+
+For the headline metric, what useless thing could the model learn and still score well (a
+condition of data collection, the class prior, prompt length)? Show the control run or the log row
+that detects it.
+
+### ex H: the scale first (large)
+
+> by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! -- Nanda
+
+Before any threshold, run the metric on a null model, a shuffled control, and the current baseline.
+
+| metric | null model | shuffled control | current baseline | ceiling the data allows | proposed threshold |
+|---|---|---|---|---|---|
+
+If a threshold has to be used before this table exists, say that it was set without a scale.
+
+### ex I: three ways it is false (large)
+
+> Excitement is evidence of bullshit: Generally, most true results are not exciting, but a fair amount of false results are. So from a Bayesian perspective, if a result is exciting and cool, it's even more likely to be false than normal! -- Nanda
+
+> If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky. -- Irpan
+
+Three ways the result can be false, each with the check that decides it. To claim A beats B: the
+baseline, the chance level, the controls, and the seed spread of one condition, as numbers with
+line references. Say whether the effect survived something it was not tuned on (a rephrased
+prompt set, a held-out dataset, another model size). Give a fresh subagent the artifact with no
+conclusion attached and show what it says. Apply the same to a negative result.
+
+### ex J: one implementation is not the idea (small)
+
+> It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Rahtz
+
+| the idea | what I ran (file:line) | one other way to run it | what a bug here would look like |
+|---|---|---|---|
+
+Say what would have to be true for the idea to be alive and your run to still fail.
+
+## Language
 
 LLMs of 2026 are trained to compress speech and use folky or humanistic language, but it's better
 for the agent (and user) to move toward field standard language, it's precise instead of ambiguous
@@ -232,224 +314,36 @@ norm" is precise but lacks redundant context, "the grad norm in #1" refers to so
 can't see, while "the grad norm of the kl loss in the 2nd part of training" is precise while
 reminding the user of lots of relevant context in their own language. - wassname
 
-Even a careful writer has to flag their own overloaded terms as they go:
-
-> I warn you that the "Understanding" in the title of this section is overloaded since very often we don't really understand why certain types of spikes happen. Here "understanding" refers to recognizing various patterns. -- Bekman
-
-> We should not assume two conditional hyperparameters are the same just because they have the same name! [...] the conditional hyperparameter called `learning_rate` is a *different* hyperparameter for `optimizer="Nesterov_momentum"` versus `optimizer="Adam"`. [...] the range of values that work well in each of the optimizers is typically different by several orders of magnitude. -- Godbole, Dahl, Gilmer, Shallue and Nado
-
-> And make sure it's clear which metrics you are using. For instance, if you report F-scores, be clear whether this is F1, or some other balance between precision and recall. If you report AUC, indicate whether this is the area under the ROC curve or the PR curve. -- Lones
-
-## ex #1 read the log end to end (small)
-
-> Switching from experimenting a lot and thinking a little to experimenting a little and thinking a lot was a key turnaround in productivity. When debugging with long iteration times, you really need to *pour* time into the hypothesis-forming step - thinking about what all the possibilities are, how likely they seem on their own, and how likely they seem in light of everything you've seen so far. -- Rahtz
-
-Rahtz was arguing against his own earlier habit, which was that with fast feedback you can check
-the first idea that comes to mind and narrow things down faster by trying than by thinking. That
-argument does not transfer to you. An agent that checks its first idea tends to fix on it, or
-leaves a confusing mess behind, so the fast loop buys less than it looks like it does.
-<!-- CLAUDE: wassname's point, my wording. -->
-
-Read the whole log before the hypothesis-forming step. State its length. Take the config from
-the log, not from the command you meant to run. Read each metric at four points. Quote the log
-line for each cell. Show:
-
-| metric | expected | start | early | middle | end | quoted line |
-|---|---|---|---|---|---|---|
-
-An empty cell is a metric that does not exist. Add the metric before the next run.
-<!-- annoy-less: [aphoristic definition] "An empty cell is a metric that does not exist." is the
-X-is-Y epigram shape that recurs in ex #8, #14 and #15. Written by CLAUDE. -->
-
-
-## ex #2 name a second cause for the same number (small)
-
-> What I'm advocating for here is not a blind faith in the buginess of your code, but for dramatically raising the threshold at which you start thinking 'OK, I think this is correct.' -- Jones
-
-Take the one number your diagnosis depends on. Quote the code that computes it. Name one other
-cause that gives the same number. Show both. Example: a cosine near 1 can be a shared mean or
-a collapsed latent. A second metric is needed to tell which.
-
-## ex #3 read your data (small)
-
-> Manually examining 100 examples does not take long. Even if you take one minute per image, you'd be done in under two hours. These two hours could save you a month of wasted effort. -- Ng
-
-> Read your data. Often, the quality of the data is a crucial driver of the results of your experiments. Often, it is quite bad. -- Nanda
-
-Show the first training example and the first evaluation example as the model sees them, with
-special tokens and the loss mask visible. Then show one complete output per arm, side by side,
-and the first token where they differ. Select the examples at random and say how. Add the best
-example, the worst example, and any example that looks wrong.
-
-## ex #4 chase the weird thing (small)
-
-> If you ever see a plot or a behaviour that just *seems weird*, chase right after it! Do not - do *not* - just 'hope it goes away'. Chasing anomalies is one of the most powerful ways to debug your system, because if you've noticed a problem without having had to go look for it, that means it's a *really big problem*. -- Jones
-
-Show one row per prediction recorded before the run: supported, contradicted, or unresolved,
-with the observation that decided it. Then list each behaviour that seems weird, including the
-ones you would prefer to ignore. End each line with "explained: ..." or "chasing now".
-
-## ex #5 list the options you have (small)
-
-> Build it up as you go, don't think you can build it ahead of time. Be focused on a strong mental model of what options you have (including architectural changes and losses) that you think should affect what metrics in the logs. -- wassname
-
-Keep one table in the repo. Add or correct rows before each run. Show the table:
-
-| option (architecture, loss, data, optimiser) | metric it should affect | direction and order | what separates it from the other options |
-|---|---|---|---|
-
-Give at least three options, one architectural and one loss. Say which options you change in
-this run and why. You can change several options in one run if each option has its own metric.
-Show the config diff against the run you will compare to.
-
-## ex #6 write down what you expect to see (small)
-
-> Before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possibility and brainstorm the cheapest tests that may narrow them down. -- wassname
-
-Show:
-
-| risky part | what I expect to see | too weak | too strong | buggy | metric exists? |
-|---|---|---|---|---|---|
-
-Add each metric whose last column says no. For each pass threshold, show the ceiling the data allows
-and check that the threshold is below the ceiling. Follow the job so that its finish wakes you.
-
-## ex #7 multiple diagnoses with % bets (large)
-
-> When their RL implementation doesn't work, people are often keen to either (a) adjust their network architecture or (b) adjust their hyperparameters. On the other hand, they're reluctant to say they've got a bug. Most often, it turns out they've got a bug. -- Jones
-
-> The default state of the world is that your research is false, because doing research is hard. -- Nanda
-
-Show three or more diagnoses. For each, give a credence, the strongest evidence for, and the
-strongest evidence against. One diagnosis is a bug in the code and one is a bug in the
-evaluation. Keep some credence on unknown. If a diagnosis has no evidence against it, mark it
-untested. Then give a fresh subagent the code and the log with no diagnosis attached, and ask
-for the top bugs and misconceptions. Show its list, including "found nothing".
-
-## ex #8 three ways the result is false (large)
-
-> Excitement is evidence of bullshit: Generally, most true results are not exciting, but a fair amount of false results are. So from a Bayesian perspective, if a result is exciting and cool, it's even more likely to be false than normal! -- Nanda
-
-Show three ways the result can be false, each with the check that decides it. To claim A beats
-B, give the baseline, the chance level, and the seed spread of one arm. One seed per arm is
-unresolved. Give a fresh subagent the artifact with no conclusion attached and show what it
-says. Apply the same to a negative result: a bad row is a bug until the log shows otherwise.
-<!-- annoy-less: [aphorism x2] "One seed per arm is unresolved." and "a bad row is a bug until the
-log shows otherwise" are both CLAUDE epigrams. Keep one at most. -->
-
-
-## ex #9 compare against a reference implementation (large)
-
-> We find that implementation differences which are often not reflected in publications can have dramatic impacts on performance. -- Henderson
-
-> If you are stuck, find a working reference implementation and compare it to yours. If nothing jumps out, try a bisection search: adapt their code wholesale, then half their features, and so on. -- wassname
-
-Search for reference implementations of the nearest method. Rank them by the GitHub signals:
-proof it runs (CI, a results table, a replication note), more than one human contributor, more
-than a few stars, a README with evaluation details, and links to other repos that use it. Take
-the top one, or write "no reference exists". Show:
-
-| feature | theirs (file:line) | mine | same? |
-|---|---|---|---|
-
-Include algorithm tweaks, engineering tricks, hyperparameters, and logged metrics. Give a fresh
-subagent the module and ask for at least one bug.
-
-## ex #10 localise the error (small)
-
-> The problem with using the loss curve as an indicator of correctness is somewhat that it's not reliable, but mostly because it doesn't localise errors. The shape of your loss curve says very little about where in your code you've messed up. -- Jones
-
-At the step that looks wrong, show the loss per term and the gradient norm per module. Name the
-module the error localises to.
-
-## ex #11 read the rows before the spike (small)
-
-> As you can see it's the previous frames that we need to look into when the numbers start going into very large for fp16 numbers. -- Bekman
-
-For each spike or collapse, show the log rows before it. Say which column moved first.
-
-## ex #12 name what else could score well (small)
-
-> The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al., whose pneumonia model scored AUC 0.931 in its own hospitals and 0.815 in someone else's
-
-> The model was able to correctly predict who would receive grants over 95% of the time. Apparently meaningless identifier columns were the most important predictors. [...] It turned out that in practice, the university only filled out much of this information *after* a grant application was accepted. -- Howard and Gugger
-
-For the headline metric, name one useless thing the model can learn and still score well, for
-example a condition of data collection or the class prior. Show the control arm or the row that
-detects it.
-
-## ex #13 pseudocode and external review (large)
-
-> Summarise your concept and pseudocode and do an external review in scientist mode. Perhaps describe the forward and backward pass as mermaid too. -- wassname
-
-Before a design change, or for a run you cannot explain, write the concept in plain English,
-the pseudocode with tensor shapes and parameter counts per module, and a mermaid diagram of the
-forward pass and the backward pass. Show all three. Use an available review skill or a blind
-subagent from another model family. Give it only the complete description, not your diagnosis,
-and show its verdict. If neither is available, state that in the report.
-
-## ex #14 one implementation is not the idea (small)
-
-> Trying an experiment and seeing it fail gives little information by itself. When an experiment fails, it is tempting to conclude "I tried X and it didn't work". However, if X is a high-level conceptual approach, then a more correct conclusion is "I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work". -- Steinhardt
-
-> It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Rahtz
-
-Before you call an idea dead, show the implementation you actually ran and one other
-implementation of the same idea that you did not run. Say what would have to be true for the
-idea to be alive and your run to still fail. Then do ex #7 on your own code before you
-write the negative up.
-
-| the idea | what I ran (file:line) | one other way to run it | what a bug here would look like |
-|---|---|---|---|
-
-One attempt is untested, not negative. Say which of the two this is.
-<!-- annoy-less: [#9 negative framing + aphorism] "One attempt is untested, not negative." is an
-X-not-Y one-line closer. It is the point of the exercise, so it may earn its place, but it is
-CLAUDE's line, not Steinhardt's. -->
-
-
-## ex #15 get the scale before the threshold (large)
-
-> A valuable intuition to have in mind is that, by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! Baselines are one way to get context to compare against. -- Nanda
-
-> In most cases, we do not know a priori what the intended behavior of the algorithm is. [...] If we train a neural network on a new classification task and it achieves 5 percent test error, we have no straightforward way of knowing if this is the expected behavior or suboptimal behavior. -- Goodfellow, Bengio and Courville
-
-Before you set a pass threshold, get the scale first. Run the metric on a null
-arm, a shuffled or permuted control, and the existing baseline, then set the bar against those.
-
-| metric | null arm | shuffled control | current baseline | ceiling the data allows | proposed gate |
-|---|---|---|---|---|---|
-
-A threshold chosen before this table is a number you made up. Say so if you have to use one anyway.
-<!-- annoy-less: [aphoristic closer] "A threshold chosen before this table is a number you made up." is
-a punchy section-ending epigram, the third of its kind in the exercises. Written by CLAUDE. -->
-
+Keep the list in `docs/JARGON.md` when working in a long loop.
 
 ## Reference
 
-Sources and more quotes: [README.md](README.md). Longer material, open the one you need:
-
-- [PLAYBOOK.md](PLAYBOOK.md) -- mental models, component isolation, baseline ladder, what to log, symptom tables.
-- [references/checklist.md](references/checklist.md) -- Lones's 36 do/don'ts.
-- [references/diagnostics.md](references/diagnostics.md) -- snippets: init loss, overfit one batch, gradient flow, NaN hooks, leakage tracer.
-- [references/static_analysis.md](references/static_analysis.md) -- grep patterns for silent bugs.
-- [references/loss_surface.md](references/loss_surface.md) -- visualise a custom loss and its gradient field.
-- [references/metric_stuck.md](references/metric_stuck.md) -- why a metric will not move, structural ceiling check.
-- [references/sweeps.md](references/sweeps.md) -- paired comparison and cross-seed reliability.
-- [references/llm_judges.md](references/llm_judges.md) -- judge biases, repeat draws, paired differences.
-- [references/llm_judge_litreview.md](references/llm_judge_litreview.md) -- the papers behind the judge advice.
-- [references/time_series.md](references/time_series.md) -- temporal evaluation and causal missing values.
-- [references/research_taste.md](references/research_taste.md) -- patience, information gain, de-risking.
-- [references/transformers.md](references/transformers.md) -- full traces, warmup, train-deploy parity, steering.
-- [rl/SKILL.md](rl/SKILL.md), [pinn/SKILL.md](pinn/SKILL.md) -- domain specifics. These two are
-  also skills in their own right, `ml-debug-rl` and `ml-debug-pinn`, so an agent that scans
-  subdirectories can load one on its own.
+- [PLAYBOOK.md](PLAYBOOK.md): mental models, component isolation, baseline ladder, what to log,
+  symptom tables (candidate routes, not prescriptions).
+- [references/diagnostics.md](references/diagnostics.md): init loss, overfit one batch, gradient
+  flow, NaN hooks, leakage tracer.
+- [references/static_analysis.md](references/static_analysis.md): grep patterns for silent bugs.
+- [references/sweeps.md](references/sweeps.md): paired comparison and cross-seed reliability.
+- [references/llm_judges.md](references/llm_judges.md) and
+  [references/llm_judge_litreview.md](references/llm_judge_litreview.md): judge biases and the
+  papers behind the advice.
+- [references/metric_stuck.md](references/metric_stuck.md),
+  [references/loss_surface.md](references/loss_surface.md),
+  [references/time_series.md](references/time_series.md),
+  [references/transformers.md](references/transformers.md),
+  [references/research_taste.md](references/research_taste.md),
+  [references/checklist.md](references/checklist.md).
+- [rl/SKILL.md](rl/SKILL.md), [pinn/SKILL.md](pinn/SKILL.md): domain specifics, also loadable as
+  `ml-debug-rl` and `ml-debug-pinn`.
 
 ## Sign off
 
-End your reply with one quote from this skill, in ASCII art speech balloon, said by an animal of
-your choice. Not a cow: cowsay is taken. Draw it yourself, do not run a program. Name who said the
-quote, so the reader can go and find the rest of it. Give the animal a unicode tool to hold
-(🔧 🔍 🪛 🧪 ...), pick one that fits the exercise you did.
+Before writing "looks fine", "works", "no effect", or "found the bug", paste the log lines that
+show it. Then choose one random line without loading the whole file: `shuf -n 1 fortune.txt`.
+End the reply with it as a clearly separate, random ASCII speech-balloon sign-off, said by an
+animal of your choice other than a cow, drawn by hand, holding a unicode tool that fits the
+exercise you did (🔧 🔍 🪛 🧪). Preserve the speaker attribution; where the canonical harvested
+record names only its source file, name that file instead. The fortune is not evidence for the
+diagnosis.
 
 Curated by [wassname](https://github.com/wassname).

docs/evidence/agans_debugging_9_rules.md

@@ -0,0 +1,353 @@
+# Debugging: The 9 Indispensable Rules
+
+David J. Agans
+
+> Notes: table of contents and Introduction, verbatim from a user-supplied EPUB. Extracted with `w3m -dump` on 2026-09-02; layout and images omitted. The complete book text (all 15 chapters, verbatim) is in the private dlbook repo at `agans_debugging_9_rules.md`.
+
+> Bibliographic record: *Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems*, David J. Agans, AMACOM, 2002, ISBN 978-0-8144-2678-4 (ebook). EPUB SHA-256: `ce3b6c92a7f263d0027b3b2d42c3061d06e8083d8a73de3a1f5eb523756699e4`.
+
+## Contents
+
+Contents
+
+Chapter 1: Introduction
+
+How Can That Work?
+
+Isn’t It Obvious?
+
+Anyone Can Use It
+
+It’ll Debug Anything
+
+But It Won’t Prevent, Certify, or Triage Anything
+
+More Than Just Troubleshooting
+
+A Word About War Stories
+
+Stay Tuned
+
+Chapter 2: The Rules—Suitable for Framing
+
+Chapter 3: Understand the System
+
+Read the Manual
+
+Read Everything, Cover to Cover
+
+Know What’s Reasonable
+
+Know the Road Map
+
+Know Your Tools
+
+Look It Up
+
+Remember
+
+Understand the System
+
+Chapter 4: Make It Fail
+
+Do It Again
+
+Start at the Beginning
+
+Stimulate the Failure
+
+Don’t Simulate the Failure
+
+What If It’s Intermittent?
+
+What if I’ve Tried Everything and It’s Still Intermittent?
+
+A Hard Look at Bad Luck
+
+Lies, Damn Lies, and Statistics
+
+Did You Fix It, or Did You Get Lucky?
+
+“But That Can’t Happen”
+
+Never Throw Away a Debugging Tool
+
+Remember
+
+Make It Fail
+
+Chapter 5: Quit Thinking and Look
+
+See the Failure
+
+See the Details
+
+Now You See It, Now You Don’t
+
+Instrument the System
+
+Design Instrumentation In
+
+Build Instrumentation In Later
+
+Don’t Be Afraid to Dive In
+
+Add Instrumentation On
+
+Instrumentation in Daily Life
+
+The Heisenberg Uncertainty Principle
+
+Guess Only to Focus the Search
+
+Remember
+
+Quit Thinking and Look
+
+Chapter 6: Divide and Conquer
+
+Narrow the Search
+
+In the Ballpark
+
+Which Side Are You On?
+
+Inject Easy-to-Spot Patterns
+
+Start with the Bad
+
+Fix the Bugs You Know About
+
+Fix the Noise First
+
+Remember
+
+Divide and Conquer
+
+Chapter 7: Change One Thing at a Time
+
+Use a Rifle, Not a Shotgun
+
+Grab the Brass Bar with Both Hands
+
+Change One Test at a Time
+
+Compare with a Good One
+
+What Did You Change Since the Last Time It Worked?
+
+Remember
+
+Change One Thing at a Time
+
+Chapter 8: Keep an Audit Trail
+
+Write Down What You Did, in What Order, and What Happened
+
+The Devil Is in the Details
+
+Correlate
+
+Audit Trails for Design Are Also Good for Testing
+
+The Shortest Pencil Is Longer Than the Longest Memory
+
+Remember
+
+Keep an Audit Trail
+
+Chapter 9: Check the Plug
+
+Question Your Assumptions
+
+Don’t Start at Square Three
+
+Test the Tool
+
+Remember
+
+Check the Plug
+
+Chapter 10: Get a Fresh View
+
+Ask for Help
+
+A Breath of Fresh Insight
+
+Ask an Expert
+
+The Voice of Experience
+
+Where to Get Help
+
+Don’t Be Proud
+
+Report Symptoms, Not Theories
+
+You Don’t Have to Be Sure
+
+Remember
+
+Get a Fresh View
+
+Chapter 11: If You Didn’t Fix It, It Ain’t Fixed
+
+Check That It’s Really Fixed
+
+Check That It’s Really Your Fix That Fixed It
+
+It Never Just Goes Away by Itself
+
+Fix the Cause
+
+Fix the Process
+
+Remember
+
+If You Didn’t Fix It, It Ain’t Fixed
+
+Chapter 12: All the Rules in One Story
+
+Chapter 13: Easy Exercises for the Reader
+
+A Light Vacuuming Job
+
+A Flock of Bugs
+
+A Loose Restriction
+
+The Jig Is Up
+
+Chapter 14: The View from the Help Desk
+
+Help Desk Constraints
+
+The Rules, Help Desk Style
+
+Understand the System
+
+Make It Fail
+
+Quit Thinking and Look
+
+Divide and Conquer
+
+Change One Thing at a Time
+
+Keep an Audit Trail
+
+Check the Plug
+
+Get a Fresh View
+
+If You Didn’t Fix It, It Ain’t Fixed
+
+Remember
+
+The View from the Help Desk Is Murky
+
+Chapter 15: The Bottom Line
+
+The Debugging Rules Web Site
+
+If You’re an Engineer
+
+If You’re a Manager
+
+If You’re a Teacher
+
+Remember
+
+Index
+
+## Introduction
+
+chapter
+
+1
+
+Introduction
+
+“At present I am, as you know, fairly busy, but I propose to devote my declining years to the composition of a textbook which shall focus the whole art of detection into one volume.”
+
+—SHERLOCK HOLMES, THE ADVENTURE OF THE ABBEY GRANGE
+
+This book tells you how to find out what’s wrong with stuff, quick. It’s short and fun because it has to be—if you’re an engineer, you’re too busy debugging to read anything more than the daily comics. Even if you’re not an engineer, you often come across something that’s broken, and you have to figure out how to fix it.
+
+Now, maybe some of you never need to debug. Maybe you sold your dot.com IPO stock before the company went belly-up and you simply have your people look into the problem. Maybe you always luck out and your design just works—or, even less likely, the bug is always easy to find. But the odds are that you and all your competitors have a few hard-to-find bugs in your designs, and whoever fixes them quickest has an advantage. When you can find bugs fast, not only do you get quality products to customers quicker, you get yourself home earlier for quality time with your loved ones.
+
+So put this book on your nightstand or in the bathroom, and in two weeks you’ll be a debugging star.
+
+How Can That Work?
+
+How can something that’s so short and easy to read be so useful? Well, in my twenty-six years of experience designing and debugging systems, I’ve discovered two things (more than two, if you count stuff like “the first cup of coffee into the pot contains all the caffeine”):
+
+1.  When it took us a long time to find a bug, it was because we had neglected some essential, fundamental rule; once we applied the rule, we quickly found the problem.
+
+2.  People who excelled at quick debugging inherently understood and applied these rules. Those who struggled to understand or use these rules struggled to find bugs.
+
+I compiled a list of these essential rules; I’ve taught them to other engineers and watched their debugging skill and speed increase. They really, really work.
+
+Isn’t It Obvious?
+
+As you read these rules, you may say to yourself, “But this is all so obvious.” Don’t be too hasty; these things are obvious (fundamentals usually are), but how they apply to a particular problem isn’t always so obvious. And don’t confuse obvious with easy—these rules aren’t always easy to follow, and thus they’re often neglected in the heat of battle.
+
+The key is to remember them and apply them. If that was obvious and easy, I wouldn’t have to keep reminding engineers to use them, and I wouldn’t have a few dozen war stories about what happened when we didn’t. Debuggers who naturally use these rules are hard to find. I like to ask job applicants, “What rules of thumb do you use when debugging?” It’s amazing how many say, “It’s an art.” Great—we’re going to have Picasso debugging our image-processing algorithm. The easy way and the artistic way do not find problems quickly.
+
+This book takes these “obvious” principles and helps you remember them, understand their benefits, and know how to apply them, so you can resist the temptation to take a “shortcut” into what turns out to be a rat hole. It turns the art of debugging into a science.
+
+Even if you’re a very good debugger already, these rules will help you become even better. When an early draft of this book was reviewed by skilled debuggers, they had several comments in common: Besides teaching them one or two rules that they weren’t already using (but would in the future), the book helped them crystallize the rules they already unconsciously followed. The team leaders (good debuggers rise to the top, of course) said that the book gave them the right words to transmit their skills to other members of the team.
+
+Anyone Can Use It
+
+Throughout the book I use the term engineer to describe the reader, but the rules can be useful to a lot of you who may not consider yourselves engineers. Certainly, this includes you if you’re involved in figuring out what’s wrong with a design, whether your title is engineer, programmer, technician, customer support representative, or consultant.
+
+If you’re not directly involved in debugging, but you have responsibility for people who are, you can transmit the rules to your people. You don’t even have to understand the details of the systems and tools your people use—the rules are fundamental, so after reading this book, even a pointy-haired manager should be able to help his far-more-intelligent teams find problems faster.
+
+If you’re a teacher, your students will enjoy the war stories, which will give them a taste of the real world. And when they burst onto that real world, they’ll have a leg up on many of their more experienced (but untrained in debugging) competitors.
+
+It’ll Debug Anything
+
+This book is general; it’s not about specific problems, specific tools, specific programming languages, or specific machines. Rather, it’s about universal techniques that will help you to figure out any problem on any machine in any language using whatever tools you have. It’s a whole new level of approach to the problem—for example, rather than tell you how to set the trigger on a Glitch-O-Matic digital logic analyzer, I’m going to tell you why you have to use an analyzer, even though it’s a lot of trouble to hook it up.
+
+It’s also applicable to fixing all kinds of problems. Your system may have been designed wrong, built wrong, used wrong, or just plain got broken; in any case, these techniques will help you get to the heart of the problem quickly.
+
+The methods presented here aren’t even limited to engineering, although they were honed in the engineering environment. They’ll help you figure out what’s wrong with other things, like cars, houses, stereo equipment, plumbing, and human bodies. (There are examples in the book.) Admittedly, there are systems that resist these techniques—the economy is too complex, for example. And some systems don’t need these methods; e.g., everybody already knows what’s wrong with the government.
+
+But It Won’t Prevent, Certify, or Triage Anything
+
+While this book is general about methods and systems, it’s very focused on finding the causes of bugs and fixing them.
+
+It’s not about quality development processes aimed at preventing bugs in the first place, such as ISO-9000, code reviews, or risk management. If you want to read about that, I recommend books like The Tempura Method of Totalitarian Quality Management Processes or The Feng Shui Guide to Vermin-Free Homes. Quality process techniques are valuable, but they’re often not implemented; even when they are, they leave some bugs in the system.
+
+Once you have bugs, you have to detect them; this takes place in your quality assurance (QA) department or, if you don’t have one of those, at your customer site. This book doesn’t deal with this stage either—test coverage analysis, test automation, and other QA techniques are well handled by other resources. A good book of poetry, such as How Do I Test Thee, Let Me Count the Ways, can help you while away the time as you check the 6,467,826 combinations of options in your product line.
+
+And sooner or later, at least one of those combinations will fail, and some QA guy or customer is going to write up a bug report. Next, some managers, engineers, salespeople, and customer support people will probably get together in a triage meeting and argue passionately about how important the bug is, and therefore when and whether to fix it. This subject is deeply specific to your market, product, and resources, and this book will not touch it with a ten-foot pole. But when these people decide it has to be fixed, you’ll have to look at the bug report and ask yourself, “How the heck did that happen?” That’s when you use this book (see Figure 1-1).
+
+The following chapters will teach you how to prepare to find a bug, dig up and sift through the clues to its cause, home in on the actual problem so you can fix it, and then make sure you really fixed it so you can go home triumphant.
+
+Figure 1-1. When to Use This Book.
+
+Images
+
+More Than Just Troubleshooting
+
+Though the terms are often interchanged, there’s a difference between debugging and troubleshooting, and there’s a difference between this debugging book and the hundreds of troubleshooting guides available today. Debugging usually means figuring out why a design doesn’t work as planned. Troubleshooting usually means figuring out what’s broken in a particular copy of a product when the product’s design is known to be good—there’s a deleted file, a broken wire, or a bad part. Software engineers debug; car mechanics troubleshoot. Car designers debug (in an ideal world). Doctors troubleshoot the human body—they never got a chance to debug it. (It took God one day to design, prototype, and release that product; talk about schedule pressure! I guess we can forgive priority-two bugs like bunions and male pattern baldness.)
+
+The techniques in this book apply to both debugging and troubleshooting. These techniques don’t care how the problem got in there; they just tell you how to find it. So they work whether the problem is a broken design or a broken part. Troubleshooting books, on the other hand, work only on a broken part. They boast dozens of tables, with symptoms, problems, and fixes for anything that might go wrong with a particular system. These are useful; they’re a compendium of everything that has ever broken in that type of system, and what the symptoms and fixes were. They give a troubleshooter the experience of many others, and they help in finding known problems faster. But they don’t help much with new, unknown problems. And thus they can’t help with design problems, because engineers are so creative, they like to make up new bugs, not use the same old ones.
+
+So if you’re troubleshooting a standard system, don’t ignore Rule 8 (“Get a Fresh View”); go ahead and consult a troubleshooting guide to see if your problem is listed. But if it isn’t, or if the fix doesn’t work, or if there’s no troubleshooting guide out yet because you’re debugging the world’s first digital flavor transmission system, you won’t have to worry, because the rules in this book will get you to the heart of your brand-new problem.
+
+A Word About War Stories
+
+I’m a male American electronics engineer, born in 1954. When I tell a “war story” about some problem that got solved somehow, it’s a real story, so it comes from things that male American electronics engineers born in 1954 know about. You may not be all or any of those, so you may not understand some of the things I mention. If you’re an auto mechanic, you may not know what an interrupt is. If you were born in 1985, you may not know what a record player is. No matter; the principle being demonstrated is still worth knowing, and I’ll explain enough as I go along so you’ll be able to get the principle.
+
+You should also know that I’ve taken some license with the details to protect the innocent, and especially the guilty.
+
+Stay Tuned
+
+In this book I’ll introduce the nine golden rules of debugging, then devote a chapter to each. I’ll start each chapter with a war story where the rule proved crucial to success; then I’ll describe the rule and show how it applies to the story. I’ll discuss various ways of thinking about and using the rule that are easy to remember in the face of complex technological problems (or even simple ones). And I’ll give you some variations showing how the rule applies to other stuff like cars and houses.
+
+In the final few chapters, I’ve included a set of war stories to exercise your understanding, a section on using the rules under the trying circumstances of the help desk, and a few last hints for putting what you’ve learned to work in your job.
+
+When you’re done with this book, your debugging efficiency will be much higher than before. You may even find yourself wandering around, looking for engineers in distress so you can swoop in and save the day. One bit of advice, though: Leave the leotard and cape at home.

fortune.txt

@@ -0,0 +1,313 @@
+broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task. -- Josh Achiam
+If one part is broken, the other parts can adapt and still achieve roughly acceptable performance. -- Goodfellow, Bengio and Courville
+The challenge lies in the fact that you can make these mistakes, train a model without it ever crashing, and still get a decent performance... -- Clara Sanh
+Trying an experiment and seeing it fail gives little information by itself. If X is a high-level conceptual approach, a more correct conclusion is: I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work. -- Jacob Steinhardt
+Insufficient skepticism doesn't feel like insufficient skepticism from the inside. It just feels like doing research. -- Neel Nanda
+Read your data. Often, the quality of the data is a crucial driver of the results of your experiments. Often, it is quite bad. -- Neel Nanda
+What I'm advocating for here is not a blind faith in the buginess of your code, but for dramatically raising the threshold at which you start thinking: OK, I think this is correct. -- Andy Jones
+The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al.
+Apparently meaningless identifier columns were the most important predictors. [...] the university only filled out much of this information after a grant application was accepted. -- Howard and Gugger
+Excitement is evidence of bullshit: generally, most true results are not exciting, but a fair amount of false results are. -- Neel Nanda
+If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky. -- Alex Irpan
+It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Dan Rahtz
+QUIT THINKING AND LOOK. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+CHANGE ONE THING AT A TIME. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+IF YOU DIDN'T FIX IT, IT AIN'T FIXED. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Don't let your instruments overwhelm your system. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+If you ever see a plot or a behaviour that just seems weird, chase right after it! Do not — do not — just hope it goes away. -- Andy Jones
+The cool extra functionality you were planning to write today might just magically fix this anomalous behaviour. It won't. Give up on your plan for the day and chase the anomaly instead. -- Andy Jones
+Don't be tempted to write an adaptive reward scaling scheme. It's extra nonstationarity. Just hand-scale. -- Andy Jones
+If you're new to RL, writing things from scratch is the most catastrophically self-sabotaging thing you can do. -- Andy Jones
+When their RL implementation doesn't work, people are often keen to adjust their network architecture or hyperparameters. They're reluctant to say they've got a bug. Most often, it turns out they've got a bug. -- Andy Jones
+The default state of the world is that your research is false, because doing research is hard. -- Neel Nanda
+Figuring out a system's gears takes extra work up-front, but yields dividends forever. The black-box approach is cheaper for one-off tasks, but usually doesn't yield any insights which will generalize to new tasks using the same system. -- John Wentworth
+You can't find typos in your own writing without a great deal of effort because you know what it's supposed to say. -- Gwern Branwen
+Even a single anomaly, apparently trivial in itself, can indicate the everyday mental model is not just a little bit wrong, but fundamentally wrong. -- Gwern Branwen
+Academic software is almost always a poorly-maintained kludge of leaky abstractions, awful formatting, and bugs that don't cripple things only because some other bug stops them from doing so. -- Patrick Kidger
+The first step to training a neural net is to not touch any neural net code at all and instead begin by thoroughly inspecting your data. -- Andrej Karpathy
+Manually examining 100 examples does not take long. Even if you take one minute per image, you'd be done in under two hours. These two hours could save you a month of wasted effort. -- Andrew Ng
+Overfit a single batch of only a few examples. If they do not [overfit], there is a bug somewhere and we cannot continue to the next stage. -- Andrej Karpathy
+When someone's RL implementation isn't working, people copy-paste a screenshot of their loss curve because they know they want a pretty, exponentially-decaying loss curve. The shape of your loss curve says very little about where in your code you've messed up. -- Andy Jones
+The quality ranking of candidate responses can be easily hacked by simply altering their order of appearance in the context. -- Wang et al., ACL 2024
+If there are NaNs, we should not drop them, else we end up comparing different sample sets and it's invalid. A might be a single easy sample, and B might be all 128 hard samples. Of course A looks much better, but actually it failed on the vast majority of samples. -- wassname
+All labels in your dataset are -100. Training losses will be all 0. -- Unsloth troubleshooting FAQ
+Don't just do the first experiment that pops into your head. Think about the key ways the hypothesis could be false, and how you could test that. -- Neel Nanda
+Do ablations on your fancy method. It's easy for people to have a fancy method with lots of moving parts, when many actually are unnecessary. -- Neel Nanda
+Don't reinvent the wheel. A common mistake in mech interp is doing something that's already been done. We have LLM-powered literature reviews now. You have way less of an excuse. Check first! -- Neel Nanda
+Good writing is simple. There's a tendency towards verbosity or trying to make things sound more complex and fancy than they actually are, so they feel impressive. I think this is a highly ineffective strategy. -- Neel Nanda
+The standard hypothesis testing framework can be misleading: most of your probability mass should normally be on something I haven't thought of yet. -- Neel Nanda
+A perfect fit can always be obtained by using a model with enough parameters. Over-fitting a model to data is just as bad as failing to identify a systematic pattern in the data. -- Hyndman and Athanasopoulos, *Forecasting: Principles and Practice*
+We made exactly the same mistake in one of my projects on insect recognition. [...] The learned classifier was surprisingly good. But a saliency map revealed that it was reading the bubble patterns and ignoring the specimens. I was so embarrassed that I had made the oldest mistake in the book. Lesson: always randomize even if you don't know what you are controlling for! -- Thomas G. Dietterich, quoted in Gwern's *Tank* evidence collection
+The entropy of your policy network's outputs usually starts near 1, then rapidly falls for a while, then flattens out for the rest of training. If it drops to zero, your agent has collapsed into some — likely myopic — policy, and isn't exploring any more. -- Andy Jones
+Bugs are just one more source of noise and your neural net is going to try its damnedest to pull the signal out of that mess you're feeding it. -- Andy Jones
+Don't try to debug your implementation by just running it on your full task. That might take days! That way madness lies. -- Andy Jones
+I missed a multithreading bug for several months by ignoring a small but mysterious decay in frames per second. -- Dan Rahtz
+Your misconfigured neural net will throw exceptions only if you're lucky; most of the time it will train but silently work a bit worse. -- Andrej Karpathy
+A fast and furious approach to training neural networks does not work and only leads to suffering. -- Andrej Karpathy
+You can't tell it's broken if you can't see that it's breaking. -- Josh Achiam
+You can think up thousands of possible reasons for a failure. You can see only the actual cause. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Don't stop when you hear the pump. Go down to the basement and find out which pump. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Build instrumentation in. Use source code debuggers, debug logs, status messages, flashing lights, and rotten egg odors. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Remove the changes that didn't do what you expected. They probably did something you didn't expect. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Compare it with a good one. If the bad ones all have something that the good ones don't, you're onto the problem. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Check that it's really your fix that fixed it. Wubba! might not be the thing that did the trick. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Know that it never just goes away by itself. Make it come back by using the original Make It Fail methods. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Fix the cause. Tear out the useless eight-track deck before you burn out another transformer. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+If you're doing anything that involves an RL algorithm as a component in a larger system, don't try and implement the RL algorithm yourself. RL is unstable enough that you'll never be sure whether your system doesn't work because of a bug in your RL implementation or because of a bug in your larger system. -- Dan Rahtz
+We find that implementation differences which are often not reflected in publications can have dramatic impacts on performance. -- Henderson et al., *Deep RL That Matters*
+When good programmers debug hard problems fast, it's usually because they understand the system well enough to track the important internal state in their head, letting them drastically reduce the solution space they're searching over. -- Ulisse Mini
+It seems important to really commit yourself to always investigate whenever you notice confusion. -- Dan Rahtz
+It turns out that bad labels are a huge problem in many popular benchmark datasets. -- Vincent Warmerdam
+Doing well on the training set is easy: just memorize the examples. The most common mistake among machine learning beginners is to test on the training data and have the illusion of success. -- Pedro Domingos
+Contamination of your classifier by test data can occur in insidious ways, for example if you use test data to tune parameters and do a lot of tuning. -- Pedro Domingos
+Most common neural net mistakes: you didn't try to overfit a single batch first; you forgot to toggle train/eval mode; you forgot to zero_grad before backward; you passed softmaxed outputs to a loss that expects raw logits. -- Andrej Karpathy
+Thinking view() and permute() are the same thing. -- Andrej Karpathy
+Rescale the rewards, but don't shift mean, as that affects agent's will to live. -- John Schulman, *Nuts and Bolts of Deep RL*
+Changing Anything Changes Everything. -- Sculley et al., *Hidden Technical Debt in Machine Learning Systems*
+Switching to the BOS dataloader changes the validation loss and makes all previous experiments not comparable in absolute value. The loss appears lower but this is fake to some extent. -- Andrej Karpathy, nanochat experiment log
+The spikes usually happen because of a bad data pocket, either due to badly shuffled data or because it hasn't been cleaned from some garbage scraped from the websites. -- Stas Bekman
+The best way to debug an error that arises in trainer.train() is to manually go through this whole pipeline to see where things went awry. The error is then often very easy to solve. -- Hugging Face course
+Hyperparameter tuning is always emphasized as being the hardest part of machine learning, but it's just the last step to help you gain a little bit on the metric. Don't launch into a time-consuming and costly hyperparameter search until you have something that beats the baseline. -- Hugging Face course
+Eliminate concurrency: restrict the number of processes to 1 for both training and data preprocessing. -- Axolotl debugging guide
+How reliable is my experiment? Ask yourself: How surprised would I be if it turned out to be complete bullshit due to a bug, error, noise, misunderstanding, etc.? Investigate the most uncertain bits. -- Neel Nanda
+Actively seek alternatives: what are the simplest explanations? What known circuits or phenomena could be involved? What would a strong skeptic argue? -- Neel Nanda
+This doesn't seem like it will work or I feel less motivated after trying a few things along this line that didn't work are not ruling out an idea. -- Jacob Steinhardt
+I had all the data necessary to make this realization a couple weeks in but had failed to do so. -- Jacob Steinhardt
+Most importantly, there is no point of launching 1000 runs with different hyperparameters: it is delusional to expect to get your biggest jumps of performance by simply tuning a few values. -- Clara Sanh
+Third, and perhaps most important for building skill, you must notice when you're going into brute-force search mode, and then take action by investing time in understanding the underlying system. -- Ulisse Mini
+Pro-tip: when you work with language, have a serious look at the outputs of the tokenizers. I can't count the number of lost hours I spent trying to reproduce results because something went wrong with the tokenization. -- Clara Sanh
+Error analysis can often help you figure out how promising different directions are. It might result in your team spending a month only to realize afterward that it resulted in little benefit. -- Andrew Ng
+If you are doing distributed training, print samples of your dataset in each process and triple-check that you get the same thing. -- Hugging Face course
+A valuable intuition: by default, all numbers are meaningless because we lack any scale to compare them. -- Neel Nanda
+If the loss or metric on your initial model is very different from the value you expect for random predictions, double-check how your loss or metric is computed: there is probably a bug there. -- Hugging Face course
+If a machine learning model can become state of the art by squeezing another 0.5% out of a validation set one has to wonder: are we really making a better model? Or are we creating a model that is better able to overfit on the bad labels? -- Vincent Warmerdam
+Most numerical errors manifest as all your metrics going weird at the same time: your loss exploding, your KL div collapsing, your rewards oscillating. From the outside, you can tell something is wrong but you've no idea what is wrong or where to start looking. -- Andy Jones
+If you arrive in RL expecting a garbage fire, you might just stay zen throughout. -- Andy Jones
+Iteration speed is a huge determinant of debugging speed. Running a test should take at most as long as it takes you to make a potential fix: a few seconds. -- Andy Jones
+Find tests that cut your system in half in some way, and tell you which half the problem is in. -- Andy Jones
+The wise thing to do is to look under the streetlight, or to look in the dark. Best moral I've heard for it is: it depends. -- Andy Jones
+Make sure you can walk before you try running. -- Andy Jones
+If it doesn't work, assume there's a bug. Spend a lot of effort searching for bugs before you resort to tweaking hyperparameters: usually it's a bug. -- Josh Achiam
+Sometimes things will work in one environment even when you have a breaking bug. -- Josh Achiam
+Measure everything. Do a lot of instrumenting to see what's going on under-the-hood. -- Josh Achiam
+Backprop plus SGD does not magically make your network work. Batch norm does not magically make it converge faster. And just because you can formulate your problem as RL doesn't mean you should. -- Andrej Karpathy
+If you insist on using the technology without understanding how it works you are likely to fail. -- Andrej Karpathy
+What we try to prevent very hard is the introduction of a lot of unverified complexity at once, which is bound to introduce bugs or misconfigurations that will take forever to find, if ever. -- Andrej Karpathy
+The unambiguously correct place to visualize your data is immediately before y_hat = model(x). This is the only source of truth. -- Andrej Karpathy
+It is a depressing fact that your network will typically still train okay because it will learn to ignore data from the other examples. -- Andrej Karpathy
+You will have hypotheses that are wrong, experiments that are inconclusive, beautiful methods that lose to dumb baselines, etc. This is totally fine and normal. -- Neel Nanda
+It is easy to be sloppy in the name of speed and introduce many bugs that cost you time in the long-run. -- Neel Nanda
+LLM-generated evaluators simply inherit all the problems of the LLMs they evaluate, requiring further human validation. -- Shankar et al.
+A ruler in a biopsy image can be correlated with malignancy because dermatologists use rulers for lesions that are a cause for concern. The algorithm doesn't know why, so it could misinterpret a random ruler sighting as grounds to diagnose cancer. -- Ricardo Novoa, quoted in Gwern's *Tank* evidence collection
+Rewarding each timestep without the pancake on the floor teaches the agent to hurl the pancake into the air as hard as possible. -- Christine Barron, quoted in Gwern's *Tank* evidence collection
+If you've learned nothing in 2 hours, pivot to another approach. If 2–3 approaches were dead ends, it's fine to just pick another problem. -- Neel Nanda
+It's all in the log. Well, the instrumentation is in the log, but what the tester saw and didn't like is not. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+They were ready to take him to the loony bin, when they noticed he wasn't wearing shoes. While he may be accused of being insane for working in a hardware lab with bare feet, he wasn't hallucinating about the bug. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+A problem with garbage characters proved to be correlated with the times that Fred was on duty. It turns out that Fred had a big gut, which would press on the keyboard when he reached up for the coffeepot. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Never trust your memory with a detail — write it down. The details you didn't think were important will prove to be the critical ones. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+The horror of that moment, the King went on, I shall never, never forget! You will, though, the Queen said, if you don't make a memorandum of it. -- Lewis Carroll, quoted by David J. Agans
+Just because you pay people $50 an hour doesn't mean that they know how to debug something. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+When you think you've fixed an engineering design, take the fix out. Make sure it's broken again. Put the fix back in. Make sure it's fixed again. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Everyone wants to believe that the bug just went away. Guess what? It will. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+If you have to ship it, ship it with a trap to catch it when it happens in the field. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Logs and other system-generated audit trails are much more reliable than users. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+When users report an error, they often give you the answer they assume is true instead of looking at the failure. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Reassembling any more than is absolutely necessary before testing makes it probable that you have not fixed the problem and will have to disassemble everything again, with a probability that increases in proportion to the amount of reassembly effort involved. -- Goldberg's Corollary to Murphy's Law, quoted by David J. Agans
+You may expect a wiring error to stop a terminal from ever working. It might work poorly because an unconnected blue wire and purple wire coupled enough signal across a hundred feet of cable. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Divide and Conquer is the only rule that actually involves finding the problem. All the others are just to help you follow this one. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+See the failure. The senior engineer saw the real failure and was able to find the cause. The junior guys thought they knew what the failure was and fixed something that wasn't broken. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Guess only to focus the search. Go ahead and guess that the memory timing is bad, but look at it before you build a timing fixer. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Grab the brass bar with both hands. If you try to fix the nuke without knowing what's wrong first, you may have an underwater Chernobyl on your hands. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+Don't assume that it was the wires and send that dirty fuel filter back onto the road. -- David J. Agans, *Debugging: The 9 Indispensable Rules*
+If each run takes 10 hours, you can easily waste a lot of time. Last run didn't work? OK, I think it's this thing. A week later, you still haven't solved the problem. -- Dan Rahtz
+If you have rapid feedback, you can narrow down the hypothesis space a lot faster by trying things than thinking carefully. -- Dan Rahtz
+When ruling out ideas, it is important to hold oneself to a high standard. -- Jacob Steinhardt
+If a result is exciting and cool, it's even more likely to be false than normal. -- Neel Nanda
+One common way an experiment fails is that it turns out to be more entangled than expected: all of the approaches you try might have the same underlying failure. -- Jacob Steinhardt
+Error goes up: commonly, this is due to a flipped sign somewhere in the loss function or gradient. Error explodes: usually a numerical issue, but can be a high learning rate. -- FSDL course
+Visualize the model in action. Directly observing the machine learning model performing its task will help determine whether the quantitative performance numbers it achieves seem reasonable. -- Goodfellow, Bengio and Courville
+By reaching a local optimum, learning curves can indicate successful optimization when the returns are not qualitatively representative of learning the desired behaviour. -- Henderson et al., *Deep RL That Matters*
+A graph of 7 tasks with 3 algorithms can look like one algorithm is best on all problems, but turn out to be the same algorithm with different random seeds. -- William Falcon
+The learning rate is a nuisance hyperparameter: we can only fairly compare models if it is tuned separately for each model. -- Google Tuning Playbook
+In the early stages of setting baselines I like to use Adam with a learning rate of 3e-4. In my experience Adam is much more forgiving to hyperparameters, including a bad learning rate. -- Andrej Karpathy
+The loss never went up in the first place. It was under-reporting loss due to exactly repeated data; it reached data it hadn't seen before and started reporting correctly. -- Stas Bekman
+The problem when you encounter an error in trainer.train() is that it could come from multiple sources. -- Hugging Face course
+Only when you manage to pass the overfitting test can you be sure that your model can actually learn something. -- Hugging Face course
+A clear condition that training works is that the model fits one batch, with the correct labels, at the expected loss. -- Clara Sanh
+If your loss or metric differs greatly from random predictions, check the loss function: the label can be wrong, the inputs can be wrong, or you might have a bug. -- Hugging Face course
+The standard hypothesis-testing framework has an implicit frame of being able to list all the hypotheses. But most of your probability mass should normally be on something I haven't thought of yet. -- Neel Nanda
+The first step is just making time to stop and ask yourself: do I endorse what I'm doing, and could I be doing something better? -- Neel Nanda
+Instability to random seed is like a canary in a coal mine. If pure randomness leads to this much variance between runs, imagine how much an actual difference in code could make. -- Alex Irpan
+Measure samples before the model sees them. Model inputs are the source of truth; upstream plots can lie. -- Andrej Karpathy
+UNDERSTAND THE SYSTEM MAKE IT FAIL QUIT THINKING AND LOOK DIVIDE AND CONQUER CHANGE ONE THING AT A TIME KEEP AN AUDIT TRAIL CHECK THE PLUG GET A FRESH VIEW IF YOU DIDN'T FIX IT, IT AIN'T FIXED -- curated in README.md
+**Quit Thinking and Look**: You can think up thousands of possible reasons for a failure. You can see only the actual cause. -- curated in README.md
+See the failure. The senior engineer saw the real failure and was able to find the cause. The junior guys thought they knew what the failure was and fixed something that wasn't broken. See the details. Don't stop when you hear the pump. Go down to the basement and find out which pump. Build instrumentation in. Use source code debuggers, debug logs, status messages, flashing lights, and rotten egg odors. Add instrumentation on. Use analyzers, scopes, meters, metal detectors, electrocardiography machines, and soap bubbles. Don't be afraid to dive in. So it's production software. It's broken, and you'll have to open it up to fix it. Watch out for Heisenberg. Don't let your instruments overwhelm your system. Guess only to focus the search. Go ahead and guess that the memory timing is bad, but look at it before you build a timing fixer. -- curated in README.md
+**Change One Thing at a Time**: You need some predictability in your life. Remove the changes that didn't do what you expected. They probably did something you didn't expect. -- curated in README.md
+Isolate the key factor. Don't change the watering schedule if you're looking for the effect of the sunlight. Grab the brass bar with both hands. If you try to fix the nuke without knowing what's wrong first, you may have an underwater Chernobyl on your hands. Change one test at a time. I knew my VGA capture phase was broken because nothing else was changing. Compare it with a good one. If the bad ones all have something that the good ones don't, you're onto the problem. Determine what you changed since the last time it worked. My friend had changed the cartridge on the turntable, so that was a good place to start. -- curated in README.md
+**If You Didn't Fix It, It Ain't Fixed**: And now that you have all these techniques, there's no excuse for leaving it unfixed. -- curated in README.md
+Check that it's really fixed. Don't assume that it was the wires and send that dirty fuel filter back onto the road. Check that it's really your fix that fixed it. "Wubba!" might not be the thing that did the trick. Know that it never just goes away by itself. Make it come back by using the original Make It Fail methods. If you have to ship it, ship it with a trap to catch it when it happens in the field. Fix the cause. Tear out the useless eight-track deck before you burn out another transformer. Fix the process. Don't settle for just cleaning up the oil. Fix the way you design machines. -- curated in README.md
+before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possiblity and brainstorm the cheapest tests that may narrow them down. - wassname -- curated in README.md
+Switching from experimenting a lot and thinking a little to experimenting a little and thinking a lot was a key turnaround in productivity. When debugging with long iteration times, you really need to *pour* time into the hypothesis-forming step - thinking about what all the possibilities are, how likely they seem on their own, and how likely they seem in light of everything you've seen so far. Spend as much time as you need, even if it takes 30 minutes, or an hour. Reserve experiments for once you've fleshed out the hypothesis space as thoroughly as possible and know which pieces of evidence would allow you to best distinguish between the different possibilities.[^rahtz] -- curated in README.md
+If you are stuck, find a working reference implementation and compare it to yours. Relvent as the hyperparameters, model, data but especially subtle things like algorithm tweaks, and engineering tricks. If nothing jumps out, the fastest way might be to try a bisection search. Here you adapt their code wholesale and try the quickest test you can. If their code works then try again with half their features and so on. Eventuall you narrow down the features that are nessesary - wassname -- curated in README.md
+If you're doing anything that involves an RL algorithm as a component in a larger system, don't try and implement the RL algorithm yourself. [...] RL is unstable enough at the moment that you'll never be sure whether your system doesn't work because of a bug in your RL implementation or because of a bug in your larger system.[^rahtz] -- curated in README.md
+We find that implementation differences which are often not reflected in publications can have dramatic impacts on performance.[^henderson] -- curated in README.md
+When their RL implementation doesn't work, people are often keen to either (a) adjust their network architecture or (b) adjust their hyperparameters. On the other hand, they're reluctant to say they've got a bug. Most often, it turns out they've got a bug. Why bugs are so much more common in RL code is discussed above, but there's another advantage to assuming you've got a bug: bugs are a damn sight faster to find and fix than validating that your new architecture is an improvement over the old one.[^jones] -- curated in README.md
+What I'm advocating for here is not a blind faith in the buginess of your code, but for dramatically raising the threshold at which you start thinking 'OK, I think this is correct.'[^jones] -- curated in README.md
+"If one part is broken, the other parts can adapt and still achieve roughly acceptable performance" [^goodfellow], -- curated in README.md
+The default state of the world is that your research is false, because doing research is hard.[^nanda] -- curated in README.md
+Excitement is evidence of bullshit: Generally, most true results are not exciting, but a fair amount of false results are. So from a Bayesian perspective, if a result is exciting and cool, it's even more likely to be false than normal![^nanda] -- curated in README.md
+When good programmers debug hard problems fast, it's usually because they understand the system well enough to *track the important internal state* in their head, letting them drastically *reduce the solution space they're searching over.*[^ulisse] -- curated in README.md
+figuring out a system's gears takes extra work up-front, but yields dividends forever. [...] The black-box approach is cheaper for one-off tasks, but usually doesn't yield any insights which will generalize to new tasks using the same system[^wentworth] -- curated in README.md
+broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task.[^spinningup] -- curated in README.md
+If you ever see a plot or a behaviour that just *seems weird*, chase right after it! Do not - do *not* - just 'hope it goes away'. Chasing anomalies is one of the most powerful ways to debug your system, because if you've noticed a problem without having had to go look for it, that means it's a *really big problem*. [...] It's really tempting to think that the cool extra functionality you were planning to write today [...] might just magically fix this anomalous behaviour. It won't. Give up on your plan for the day and chase the anomaly instead.[^jones] -- curated in README.md
+It was only by following that confusion and realising that taking the difference between frames zeroed out the background that gave the hint of a problem with normalization.[^rahtz] -- curated in README.md
+It seems important to really commit yourself to *always* investigate whenever you notice confusion.[^rahtz] -- curated in README.md
+you can't find typos in your own writing without a great deal of effort because you know what it's *supposed* to say; so copyediting advice runs like 'read it out loud' or 'print it out and read it' or 'wait a week' [...] or even 'read it upside down'. That's the sort of thing it takes to force you to read what you actually wrote, and not what you thought you wrote.[^gwern-unseeing] -- curated in README.md
+Academic software is almost always a poorly-maintained kludge of leaky abstractions, awful formatting, and bugs that don't cripple things only because some other bug stops them from doing so.[^kidger] -- curated in README.md
+This is a systemic professional failing. [...] the overwhelming majority of your time will be spent in front of a screen, staring at code. And yet most of you (yes, you) would not pass muster as a junior developer.[^kidger] -- curated in README.md
+When someone's RL implementation isn't working, they *luuuuuurv* to copy-paste a screenshot of their loss curve to you. They do this because they know they want a pretty, exponentially-decaying loss curve, and they know what they have *isn't that*. The problem with using the loss curve as an indicator of correctness is somewhat that it's not reliable, but mostly because it doesn't localise errors. The shape of your loss curve says very little about where in your code you've messed up, and so says very little about what you need to change to get things working.[^jones] -- curated in README.md
+The first step to training a neural net is to not touch any neural net code at all and instead begin by thoroughly inspecting your data. [...] The outliers especially almost always uncover some bugs in data quality or preprocessing.[^karpathy-recipe] -- curated in README.md
+Manually examining 100 examples does not take long. Even if you take one minute per image, you'd be done in under two hours. These two hours could save you a month of wasted effort.[^ng-mly] -- curated in README.md
+It turns out that bad labels are a *huge* problem in many popular benchmark datasets.[^koaning] -- curated in README.md
+A cautionary tale in artificial intelligence tells about researchers training an neural network (NN) to detect tanks in photographs, succeeding, only to realize the photographs had been collected under specific conditions for tanks/non-tanks and the NN had learned something useless like time of day.[^gwern] -- curated in README.md
+Doing well on the training set is easy (just memorize the examples). The most common mistake among machine learning beginners is to test on the training data and have the illusion of success.[^domingos] -- curated in README.md
+Contamination of your classifier by test data can occur in insidious ways, for example, if you use test data to tune parameters and do a lot of tuning. (Machine learning algorithms have lots of knobs, and success often comes from twiddling them a lot, so this is a real concern.)[^domingos] -- curated in README.md
+Overfit a tiny subset of data. Lastly and most importantly, before training on the full dataset try to train on a tiny portion (e.g. 20 examples) of your data and make sure you can achieve zero cost. For this experiment it's also best to set regularization to zero [...]. Unless you pass this sanity check with a small dataset it is not worth proceeding to the full dataset.[^cs231n] -- curated in README.md
+Overfit a single batch of only a few examples (e.g. as little as two). [...] If they do not, there is a bug somewhere and we cannot continue to the next stage.[^karpathy-recipe] -- curated in README.md
+most common neural net mistakes: 1) you didn't try to overfit a single batch first. 2) you forgot to toggle train/eval mode for the net. 3) you forgot to .zero_grad() (in pytorch) before .backward(). 4) you passed softmaxed outputs to a loss that expects raw logits. ; others? :)[^karpathy-mistakes] -- curated in README.md
+oh: 5) you didn't use bias=False for your Linear/Conv2d layer when using BatchNorm, or conversely forget to include it for the output layer .This one won't make you silently fail, but they are spurious parameters[^karpathy-mistakes] -- curated in README.md
+6) thinking view() and permute() are the same thing (& incorrectly using view)[^karpathy-mistakes] -- curated in README.md
+Look, there's variance in supervised learning too, but it's rarely this bad. If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky.[^irpan] -- curated in README.md
+Instability to random seed is like a canary in a coal mine. If pure randomness is enough to lead to this much variance between runs, imagine how much an actual difference in the code could make.[^irpan] -- curated in README.md
+- If observations have unknown range, standardize - Compute running estimate of mean and standard deviation - x' = clip((x - mu)/sigma, -10, 10) - Rescale the rewards, but don't shift mean, as that affects agent's will to live - Standardize prediction targets (e.g., value functions) the same way -- curated in README.md
+Always Be Ablating - Different tricks may substitute - Especially whitening -- curated in README.md
+**Entanglement.** Machine learning systems mix signals together, entangling them and making isolation of improvements impossible. For instance, consider a system that uses features x1, ...xn in a model. If we change the input distribution of values in x1, the importance, weights, or use of the remaining n − 1 features may all change. [...] No inputs are ever really independent. We refer to this here as the CACE principle: Changing Anything Changes Everything. CACE applies not only to input signals, but also to hyper-parameters, learning settings, sampling methods, convergence thresholds, data selection, and essentially every other possible tweak.[^sculley] -- curated in README.md
+Although one might think we would spend most of our time trying to maximize performance on the validation set, in practice we spend the majority of our time trying to gain insight into the problem, and comparatively little time greedily focused on the validation error. In other words, we spend most of our time on "exploration" and only a small amount on "exploitation".[^tuning-playbook] -- curated in README.md
+The learning rate is a nuisance hyperparameter because we can only fairly compare models with different numbers of hidden layers if the learning rate is tuned separately for each number of layers (the optimal learning rate generally depends on the model architecture).[^tuning-playbook] -- curated in README.md
+In the early stages of setting baselines I like to use Adam with a learning rate of 3e-4. In my experience Adam is much more forgiving to hyperparameters, including a bad learning rate.[^karpathy-recipe] -- curated in README.md
+We are nearing the point of wiping out a source of transformer training instability with one simple intervention.[^lucidrains] -- curated in README.md
+Do note that switching to the BOS dataloader changes the validation loss and makes all previous experiments not comparable in absolute value of the loss, because we have a lot fewer "confusing" tokens in the train/val batches. [...] Therefore, the loss appears lower but this is "fake" to some extent.[^nanochat] -- curated in README.md
+Original implementation clipped local gradients before sync. Since this codebase doesn't use DDP (gradient sync is in the optimizers), each rank was clipping based on its own local norm.[^nanochat] -- curated in README.md
+As you can see it's the previous frames that we need to look into when the numbers start going into very large for fp16 numbers.[^bekman] -- curated in README.md
+In general there are 3 types of loss spikes: 1. Fast recovering spikes 2. Slow recovering spikes 3. Not fully recovering spikes -- curated in README.md
+The spikes usually happen because of a bad data pocket, either due to badly shuffled data or because it hasn't been cleaned from some garbage scraped from the websites.[^bekman-book] -- curated in README.md
+We think the 2 main obstacles were using fp16 and data that had a lot of garbage in it. For BLOOM-176B we switched to bf16, used much cleaner data and also added an embedding layer-norm and that made all the difference.[^bekman-book] -- curated in README.md
+The best way to debug an error that arises in `trainer.train()` is to manually go through this whole pipeline to see where things went awry. The error is then often very easy to solve.[^hfcourse] -- curated in README.md
+Hyperparameter tuning is always emphasized as being the hardest part of machine learning, but it's just the last step to help you gain a little bit on the metric. [...] don't launch into a time-consuming and costly hyperparameter search until you have something that beats the baseline you have on your dataset.[^hfcourse] -- curated in README.md
+The most common cause of this error is using an **incorrect chat template**. It's essential to use the SAME chat template that was used when training the model in Unsloth and later when you run it in another framework, such as llama.cpp or Ollama. [...] It might also be because your inference engine adds an unnecessary "start of sequence" token (or the lack of thereof on the contrary) so ensure you check both hypotheses![^unsloth] -- curated in README.md
+All labels in your dataset are -100. Training losses will be all 0.[^unsloth] -- curated in README.md
+**Eliminate concurrency**: Restrict the number of processes to 1 for both training and data preprocessing[^axolotl] -- curated in README.md
+Axolotl caches certain steps and so does the underlying HuggingFace trainer. You may want to clear some of these caches when debugging.[^axolotl] -- curated in README.md
+4. Think your algorithm is working but you're actually seeing random noise. - Example: Graph of 7 tasks with 3 algorithms and looks like 1 algorithm might be doing best on all problems, but turns out they're all the same algorithm with DIFFERENT random seeds. -- curated in README.md
+Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research.[^nanda-mindsets] -- curated in README.md
+**The challenge lies in the fact that you can make these mistakes, train a model without it ever crashing, and still get a decent performance…**[^sanh] -- curated in README.md
+-   It is all well and good to make comparisons of validation error rates estimated on a finite validation set using fastidious statistical tests, but often the trial variance alone can produce statistically significant differences between two different trained models that use the same hyperparameter settings.[^tuning-playbook] -- curated in README.md
+**How reliable is my experiment?** Ask yourself: "How surprised would I be if it turned out to be complete bullshit due to a bug, error, noise, misunderstanding, etc.?" Investigate the most uncertain bits[^nanda-papers] -- curated in README.md
+Insufficient Skepticism: Missing simple alternative explanations, methodological flaws, or bugs. Explicitly list alternatives. Get others (especially mentors) to red team your plans before you run them. Actively try to break your hypothesis. Ask "What observation would make me abandon this?"[^nanda-taste] -- curated in README.md
+**Trying an experiment and seeing it fail gives little information by itself.** When an experiment fails, it is tempting to conclude "I tried X and it didn't work". However, if X is a high-level conceptual approach, then a more correct conclusion is "I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work".[^steinhardt] -- curated in README.md
+When ruling out ideas, it is important to hold oneself to a high standard. "This doesn't seem like it will work" or "I feel less motivated after trying a few things along this line that didn't work" are _not_ ruling out an idea.[^steinhardt] -- curated in README.md
+When a machine learning system performs poorly, it is usually difficult to tell whether the poor performance is intrinsic to the algorithm itself or whether there is a bug in the implementation of the algorithm. Machine learning systems are difficult to debug for various reasons.[^goodfellow] -- curated in README.md
+It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs?[^irpan] -- curated in README.md
+**Result:** This was not an out-of-the-box win for nanochat even with a mild attempt over a few hours at a bit of tuning and debugging. The idea itself is intuitively appealing. Might come back around later to try harder later.[^nanochat] -- curated in README.md
+Our specific recommendations to researchers include: 1. Computing standard errors of the mean using the Central Limit Theorem 2. When questions are drawn in related groups, computing clustered standard errors 3. Reducing variance by resampling answers and by analyzing next-token probabilities 4. When two models are being compared, conducting statistical inference on the question-level paired differences, rather than the population-level summary statistics 5. Using power analysis to determine whether an eval (or a random subsample) is capable of testing a hypothesis of interest[^miller] -- curated in README.md
+If you keep that strategy when each run takes 10 hours, though, you can easily waste a *lot* of time. Last run didn’t work? OK, I think it’s this thing. Let’s set off another run to check. Coming back the next morning: still doesn’t work? OK, maybe it’s this other thing. Let’s set off another run. A week later, you still haven’t solved the problem.[^rahtz] -- curated in README.md
+than forming hypotheses. Why spend 15 minutes carefully considering everything that could be causing what you see when you can check the first idea that jumps to mind in a fraction of that (and gather more evidence in the process)? To put it another way: if you have rapid feedback, you can narrow down the hypothesis space a lot faster by trying things than thinking carefully.[^rahtz] -- curated in README.md
+The standard hypothesis testing framework can be misleading here, because it has an implicit frame of being able to list all the hypotheses. But actually, most of your probability mass should normally be on “something I haven’t thought of yet”[^nanda-mindsets] -- curated in README.md
+If trying to explain something mysterious, novice researchers often neglect simple, dumb hypotheses like “maybe MLP0 is incredibly important on *every* input, and there’s nothing special going on with my prompt”[^nanda] -- curated in README.md
+Importantly, it is often not obvious that multiple approaches to a problem all have the same issue. In the past, I have spent months trying different approaches to a problem before finally stepping back and realizing that they were all failing for the same reason. Moreover, I had all the data necessary to make this realization a couple weeks in but had failed to do so.[^steinhardt] -- curated in README.md
+* **Error goes up**: Commonly, this is due to a flip sign somewhere in the loss function/gradient. * **Error explodes**: This is usually a numerical issue but can also be caused by a high learning rate. * **Error oscillates**: You can lower the learning rate and inspect the data for shuffled labels or incorrect data augmentation. * **Error plateaus**: You can increase the learning rate and get rid of regulation. Then you can inspect the loss function and the data pipeline for correctness.[^fsdl] -- curated in README.md
+Actively Seek Alternatives: Explicitly brainstorm other ways your observations could be explained. What are the simplest explanations? What known circuits or phenomena could be involved? What would a strong skeptic argue?[^nanda-taste] -- curated in README.md
+**If it doesn’t work, assume there’s a bug.** Spend a lot of effort searching for bugs before you resort to tweaking hyperparameters: usually it’s a bug. Bad hyperparameters can significantly degrade RL performance, but if you’re using hyperparameters similar to the ones in papers and standard implementations, those will probably not be the issue.[^spinningup] -- curated in README.md
+For example, perhaps you forgot to flip your labels when you left-right flipped the image during data augmentation. Your net can still (shockingly) work pretty well because your network can internally learn to detect flipped images and then it left-right flips its predictions. Or maybe your autoregressive model accidentally takes the thing it’s trying to predict as an input due to an off-by-one bug. Or you tried to clip your gradients but instead clipped the loss, causing the outlier examples to be ignored during training. Or you initialized your weights from a pretrained checkpoint but didn’t use the original mean. Or you just screwed up the settings for regularization strengths, learning rate, its decay rate, model size, etc.[^karpathy-recipe] -- curated in README.md
+Most importantly, there is no point of launching 1000 runs with different hyperparameters (or architecture tweaks like activation functions): **compare a couple of runs with different hyperparameters to get an idea of which hyperparameters have the highest impact** but in general, it is delusional to expect to get your biggest jumps of performance by simply tuning a few values. For instance, if your best performing model is trained with a learning rate of 4e2, there is probably something more fundamental happening inside your neural network and you want to identify and understand this behavior so that you can re-use this knowledge outside of your current specific context.[^sanh] -- curated in README.md
+Once the algorithm was partially working, they would attain higher performance by looking for remaining bugs, both by reviewing the code carefully, and by collecting metrics such as average policy entropy to perform sanity-checks, rather than just tune hyperparameters.[^olsson] -- curated in README.md
+Third, and perhaps most important for building skill,[[1]](https://www.lesswrong.com/posts/LTypqBMTSmRrrhb2v/how-to-get-good-at-programming#fn289bs9hi65b)you must **notice** when you're going into brute-force search mode, and then **take action** by investing time in understanding the underlying system, until both the problem and solution make sense.[^ulisse] -- curated in README.md
+Things I've tried (but maybe not systematically enough):  * Different initial LRs * Different optimizers * Different number of hidden layers/units * Shared pi/V NN body (with diff output layers) vs not * Changing amount of entropy * Adding correlated noise * Using TD residual instead of MC version * Clipping the gradient * Different gamma values -- curated in README.md
+Visualize the model in action: When training a model to detect objects in images, view some images with the detections proposed by the model displayed superimposed on the image. When training a generative model of speech, listen to some of the speech samples it produces. This may seem obvious, but it is easy to fall into the practice of looking only at quantitative performance measurements like accuracy or log-likelihood. Directly observing the machine learning model performing its task will help to determine whether the quantitative performance numbers it achieves seem reasonable. Evaluation bugs can be some of the most devastating bugs because they can mislead you into believing your system is performing well when it is not.[^goodfellow] -- curated in README.md
+By reaching a local optimum, learning curves can indicate successful optimization of the policy over time, when in reality the returns achieved are not qualitatively representative of learning the desired behaviour, as demon-strated in video replays of the learned policy 5. Therefore, it is important to show not only returns but demonstrations of the learned policy in action.[^henderson] -- curated in README.md
+2. Make sure observations usable: - See if YOU could control the system by using the same observations you give the agent. - Example: Look at preprocessed images yourself to make sure you don't remove necessary details or hinder the algorithm in a certain way. -- curated in README.md
+Pro-tip: when you work with language, have a serious **look at the outputs of the tokenizers**. I can’t count the number of lost hours I spent trying to reproduce results (and sometimes my own old results) because something went wrong with the tokenization.[^sanh] -- curated in README.md
+Error analysis can often help you figure out how promising different directions are. I’ve seen many engineers reluctant to carry out error analysis. It often feels more exciting to just jump in and implement some idea, rather than question if the idea is worth the time investment. This is a common mistake: It might result in your team spending a month only to realize afterward that it resulted in little benefit.[^ng-mly] -- curated in README.md
+⚠️ If you are doing distributed training, print samples of your dataset in each process and triple-check that you get the same thing. One common bug is to have some source of randomness in the data creation that makes each process have a different version of the dataset.[^hfcourse] -- curated in README.md
+-   Although in many cases the primary objective of our experiments only requires considering the validation error of each trial, we must be careful when reducing each trial to a single number because it can hide important details about what’s going on below the surface. -   For every study, we always look at the **training curves** (training error and validation error plotted versus training step over the duration of training) of at least the best few trials.[^tuning-playbook] -- curated in README.md
+(I missed a multithreading bug for several months by ignoring a small but mysterious decay in frames per second.)[^rahtz] -- curated in README.md
+There was no real spike in the two earlier runs. The loss never went up in the first place. In both resumes it was under-reporting loss due to an exactly repeated data and then it reached data it hasn't seen before and started reporting correctly. In other words it was overfitting and reporting a false loss.[^bekman-book] -- curated in README.md
+**Do ablations on your fancy method**: It's easy for people to have a fancy method with lots of moving parts, when many actually are unnecessary. You should always try removing one part and see if the method breaks. Do this for each part. *   For example, the [original unlearning method](https://arxiv.org/abs/2403.03218v1) in the [RMU paper](https://arxiv.org/abs/2403.03218) claimed it was based on finding a meaningful steering vector, until follow-up work found that it was just about adding a vector with really high norm that broke the model, and a random vector performed just as well.[^nanda] -- curated in README.md
+The only way to find out what needs work is to implement something quickly, -- curated in README.md
+and find out what parts break.[^cs229] -- curated in README.md
+Figure 15.5: An autoencoder trained with mean squared error for a robotics task has failed to reconstruct a ping pong ball. The existence of the ping pong ball and all its spatial coordinates are important underlying causal factors that generate the image and are relevant to the robotics task. Unfortunately, the autoencoder has limited capacity, and the training with mean squared error did not identify the ping pong ball as being salient enough to encode.[^goodfellow-ch15] -- curated in README.md
+One of the key drivers of progress in mech interp is an openness to qualitative research: summary statistics lose a ton of information. What can we learn by actually looking deeply into what's happening?[^nanda] -- curated in README.md
+1. **Test reward function standalone**: Run it outside training with known inputs to verify it returns nonzero values.[^axolotl-stability] -- curated in README.md
+In most cases, we do not know a priori what the intended behavior of the algorithm is. In fact, the entire point of using machine learning is that it will discover useful behavior that we were not able to specify ourselves. If we train a neural network on a new classification task and it achieves 5 percent test error, we have no straightforward way of knowing if this is the expected behavior or suboptimal behavior.[^goodfellow] -- curated in README.md
+A valuable intuition to have in mind is that, by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! Baselines are one way to get context to compare against.[^nanda-draft] -- curated in README.md
+You might be temped to keep track of the difference \(\mid f’\_a - f’\_n \mid \) or its square and define the gradient check as failed if that difference is above a threshold. However, this is problematic. For example, consider the case where their difference is 1e-4. This seems like a very appropriate difference if the two gradients are about 1.0, so we’d consider the two gradients to match. But if the gradients were both on order of 1e-5 or lower, then we’d consider 1e-4 to be a huge difference and likely a failure.[^cs231n] -- curated in README.md
+*   How would a random predictor perform (especially in classification problems)? Dataset can be unbalanced… *   What would the loss look like for a random predictor? *   What is (are) the best metric(s) to measure progress on my task? *   What are the limits of this metric? If it’s perfect, what can I conclude? What can’t I conclude?[^sanh] -- curated in README.md
+If the loss/metric you get on your initial model is very different from the loss/metric you would expect for random predictions, double-check the way your loss or metric is computed, as there is probably a bug there. If you are using several losses that you add at the end, make sure they are of the same scale.[^hfcourse] -- curated in README.md
+5.   **Rule of thumb: 400 episodic return in breakout**: Check if your PPO could obtain 400 episodic return in breakout. We have found this to be a practical rule of thumb to determine the fidelity of online PPO implementations in GitHub. Often we found PPO repositories not able to do this, and we know they probably do not match all implementation details of `openai/baselines`’ PPO.[^ppo37] -- curated in README.md
+The issue here isn't just that we might have bad labels in our training set, the issue is that it appears in the validation set. If a machine learning model can become state of the art by squeezing another 0.5% out of a validation set one has to wonder. Are we really making a better model? Or are we creating a model that is better able to overfit on the bad labels?[^koaning] -- curated in README.md
+broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task. -- Achiam -- curated in SKILL.md
+If one part is broken, the other parts can adapt and still achieve roughly acceptable performance -- Goodfellow, Bengio and Courville -- curated in SKILL.md
+Although one might think we would spend most of our time trying to maximize performance on the validation set, in practice we spend the majority of our time trying to gain insight into the problem -- Godbole, Dahl, Gilmer, Shallue and Nado -- curated in SKILL.md
+Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research. -- Nanda -- curated in SKILL.md
+Read your data. Often, the quality of the data is a crucial driver of the results of your experiments. Often, it is quite bad. -- Nanda -- curated in SKILL.md
+How would a random predictor perform (especially in classification problems)? [...] What would the loss look like for a random predictor? [...] What are the limits of this metric? If it's perfect, what can I conclude? What can't I conclude? -- Sanh -- curated in SKILL.md
+**NEVER STOP**: Once the experiment loop has begun (after the initial setup), do NOT pause to ask the human if you should continue. Do NOT ask 'should I keep going?' or 'is this a good stopping point?'. The human might be asleep, or gone from a computer and expects you to continue working *indefinitely* until you are manually stopped. You are autonomous. If you run out of ideas, think harder — read papers referenced in the code, re-read the in-scope files for new angles, try combining previous near-misses, try more radical architectural changes. The loop runs until the human interrupts you, period. -- Karpathy, [autoresearch/program.md](https://github.com/karpathy/autoresearch/blob/master/program.md) -- curated in SKILL.md
+Build it up as you go, don't think you can build it ahead of time. Be focused on a strong mental model of what options you have (including architectural changes and losses) that you think should affect what metrics in the logs. -- wassname -- curated in SKILL.md
+Before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possibility and brainstorm the cheapest tests that may narrow them down. -- wassname -- curated in SKILL.md
+If you are stuck, find a working reference implementation and compare it to yours. If nothing jumps out, try a bisection search: adapt their code wholesale, then half their features, and so on. -- wassname -- curated in SKILL.md
+Summarise your concept and pseudocode and do an external review in scientist mode. Perhaps describe the forward and backward pass as mermaid too. -- wassname -- curated in SKILL.md
+The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al. -- curated in SKILL.md
+Apparently meaningless identifier columns were the most important predictors. [...] the university only filled out much of this information *after* a grant application was accepted. -- Howard and Gugger -- curated in SKILL.md
+by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! -- Nanda -- curated in SKILL.md
+If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky. -- Irpan -- curated in SKILL.md
+It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Rahtz -- curated in SKILL.md
+Don't be tempted to write an adaptive reward scaling scheme. It's extra nonstationarity. Just hand-scale. -- Andy Jones -- curated in rl/SKILL.md
+If you're new to RL, writing things from scratch is the most catastrophically self-sabotaging thing you can do. -- Andy Jones -- curated in rl/SKILL.md
+Rathore et al. 2024: "the estimate of the κ grows polynomially with nres" -- but this is in raw units. Nondimensionalization reduces the effective condition number by making all PDE coefficients O(1). -- curated in pinn/SKILL.md
+Wang et al. propose a modified MLP with multiplicative interactions. With `U = φ(XW1 + b1)`, `V = φ(XW2 + b2)` two nonlinear encodings of the input (φ = tanh) and a per-layer gate `Z(k) = φ(H(k)Wz,k + bz,k)` computed from the hidden state, the update is `H(k+1) = (1 - Z(k)) * U + Z(k) * V`. Authors claim a ~3x decrease in the leading Hessian eigenvalue. -- curated in pinn/SKILL.md
+Factorize each neuron's weight vector as w = s * w_unit, where s is a trainable scalar and w_unit is the unit-normalized direction. This changes the optimization geometry so the loss surface has better-conditioned local minima. "Predictions obtained by RWF are in excellent agreement with ground truth, while other weight parameterizations result in poor or non-physical approximations." -- curated in pinn/SKILL.md
+Used in the PirateNet architecture alongside causal training, sequence-to-sequence, and Fourier features. Simple to implement as a custom parameterization on Linear layers. -- curated in pinn/SKILL.md
+Instead of data-augmenting with transformed copies, bake symmetries directly into the architecture so every model in the function space is automatically invariant/equivariant. For turbulence closure (Reynolds stress from velocity gradients), custom tensor layers enforce Galilean invariance by construction. "The Galilean invariant model is more accurate than the other models" and generalizes better across flow configurations. -- curated in pinn/SKILL.md
+Lecture: Brunton, S. "AI/ML+Physics Part 3 - Designing an Architecture." https://www.youtube.com/watch?v=fiX8c-4K0-Q Key distinction: invariance (output unchanged by transformation, e.g., energy is frame-invariant) vs equivariance (output transforms same way as input, e.g., stress tensor rotates with frame). Equivariant architectures are more general. If your PDE has known symmetries (translation, rotation, scaling), enforce them architecturally rather than hoping the optimizer discovers them. **Caveat**: This works best for local closure terms (Reynolds stress, turbulence models) and unbounded/periodic domains where the global symmetry holds everywhere. If your domain has boundary conditions that break the symmetry (e.g., a wall breaks rotational invariance), enforcing the symmetry globally in the architecture will prevent the solution from satisfying the BCs -- the architecture will be fighting the problem. In bounded domains, use symmetry-enforcing architectures only for terms where the symmetry genuinely holds (e.g., the constitutive relation), not for the full solution field. Libraries like `e3nn` implement this but add significant computational overhead. -- curated in pinn/SKILL.md
+Rathore et al. 2024 (ICML, credence ~80%): "Adam+L-BFGS attains 14.2x smaller L2RE than Adam on convection and 6.07x smaller than L-BFGS on wave." Tested on 3 PDEs (convection, reaction, wave), 5 seeds, widths 50-400. -- curated in pinn/SKILL.md
+"on the convection PDE, a loss of 10^-3 yields an L2RE around 10^-1, but decreasing the loss by a factor of 100 to 10^-5 yields an L2RE around 10^-2, a 10x improvement." -- curated in pinn/SKILL.md
+"L-BFGS stops in these cases without reaching a critical point: the gradient norm is around 10^-2 or 10^-3. The gradient still contains useful information for improving the loss." -- curated in pinn/SKILL.md
+Cause: strong Wolfe line search fails, step size goes to zero. Fix: switch to NNCG (Armijo only) or restart with different LR. -- curated in pinn/SKILL.md
+Theorem 8.4 (Section 8.2): condition number = Omega(nres^alpha) with alpha > 1/2, given eigenvalues of A o K_inf decaying as O(j^-2alpha). nres typically ranges 1e3 to 1e4. Separately, measured condition numbers near a solution are often > 1e4 (Section 6.2, Figure 3). -- curated in pinn/SKILL.md
+L2 norm (MSE) on residuals: default; promotes smooth, low-frequency solutions. L1 norm (MAE) on residuals: more robust to outlier collocation errors and sharp gradients (shocks) since it doesn't square-penalize large pointwise residuals. This is distinct from L1 *regularization on equation coefficients*, which is what SINDy and sparse equation discovery use to promote parsimony (few active terms). Don't conflate the two: L1 residual = robust fitting; L1 coefficient regularization = sparse model selection. For standard PINNs with a known PDE, L2 is correct. L1 residual loss is worth trying if you have shocks or suspect outlier collocation points. -- curated in pinn/SKILL.md
+Wang et al. 2021 (credence ~80%): "the gradients corresponding to the boundary loss term Lub(θ) in each layer are sharply concentrated around zero and overall attain significantly smaller values than the gradients corresponding to the PDE residual loss Lr(θ)." Shown via per-layer histograms of back-propagated gradients; the paper does not quantify the gap in orders of magnitude. -- curated in pinn/SKILL.md
+Wang et al. 2021: "many eigenvalues of the residual-loss Hessian are extremely large up to 1e5" while the boundary-loss Hessian eigenvalues stay small, so the gradient-flow stiffness is dominated by the residual term. This is an absolute magnitude, not a condition number; Wang never reports one. -- curated in pinn/SKILL.md
+For a condition number, use Rathore Figure 3: outlier eigenvalues > 1e4 (convection), > 1e3 (reaction), > 1e5 (wave). -- curated in pinn/SKILL.md
+Adaptively weight each loss term inversely proportional to its gradient magnitude. EMA of gradient statistics for stability. -- curated in pinn/SKILL.md
+NeuralPDE.jl implements this as `GradientScaleAdaptiveLoss`. -- curated in pinn/SKILL.md
+Instead of summing loss gradients (which can cancel), project them into a conflict-free direction. ConFIG: unit-normalize per-loss gradients, solve least-squares for combined direction, rescale by projection lengths. -- curated in pinn/SKILL.md
+Key: must compute per-loss gradients separately (zero_grad + backward for each). Summing raw losses defeats the purpose. M-ConFIG: momentum variant, updates only one loss's gradient per step. Use with SGD, not Adam (momentum conflict). -- curated in pinn/SKILL.md
+Standard PINNs use penalized (soft) constraints: add physics as a loss term. The alternative is constrained optimization: minimize data error while exactly satisfying the physics constraints. "With a loss function you're not exactly satisfying your constraints. With constrained optimization you are." -- curated in pinn/SKILL.md
+Physics-informed DMD (Baddoo et al. 2021) is the cleanest example: restrict the DMD matrix to a symmetry-preserving manifold (Hermitian, symplectic, etc.) via the Procrustes problem. KKT closed-form solutions exist because DMD is linear in its parameters -- the constraint is linear in both the output and the parameters simultaneously. Baddoo et al. 2021. "Physics-informed dynamic mode decomposition." Proc. R. Soc. A. https://arxiv.org/pdf/2112.04307 **Critical caveat for PINNs**: A BC like u(0)=0 is affine in the output u, but it is nonlinear in the NN weights theta. Closed-form KKT does NOT apply to neural network parameters. For NN-based PINNs, the two options for hard constraints are: (a) architectural -- multiply output by a distance function that satisfies the BC (Section 4 item 8), or (b) Augmented Lagrangian Methods (ALM), which are iterative and substantially more complex than Adam. Constrained optimization is most practical for linear models (DMD, SINDy, linear state-space) where the parameters enter linearly. -- curated in pinn/SKILL.md
+When the PINN fails on hard PDE regimes (high convection coefficient, strong reaction), don't start there. Start with easy parameters (small coefficient), train to convergence, then warm-start and increase to the target regime. 1-2 orders of magnitude improvement over naive training. "The curriculum training approach achieves significantly better errors, as well as lower variance in the error." (From Figure E.2 showing 10 seeds) -- curated in pinn/SKILL.md
+For time-dependent PDEs: train on a short time window, predict next state, step forward. Don't train on full space-time at once. "Posing the problem as seq2seq learning results in significantly lower error. The difference is particularly striking for reaction and reaction-diffusion cases, where seq2seq decreases error by almost two orders of magnitude." -- curated in pinn/SKILL.md
+NeuralPDE.jl calls this time-marching; see `WeightedIntervalTraining`. Note: these failures are not due to limited NN expressivity -- the architecture has enough capacity. The problem is optimization difficulty from the soft PDE constraint. -- curated in pinn/SKILL.md
+Standard PINNs trained by gradient descent are implicitly biased toward minimizing residuals at *later* times before even fitting the initial conditions -- violating physical causality. The NTK analysis shows the residual at time t is influenced more by residuals at later t' > t than earlier ones. This makes PINNs fail on chaotic/turbulent systems. Fix: weight each temporal residual point by wi = exp(-epsilon * sum_j<i R_j(theta)), where R_j is the accumulated residual before time i. This forces earlier times to converge first before the loss "turns on" at later times. "10-100x improvements in accuracy compared to competing approaches. First time PINNs succeeded on chaotic Lorenz, Kuramoto-Sivashinsky, and 2D Navier-Stokes in turbulent regime." -- curated in pinn/SKILL.md
+Key difference from seq2seq/curriculum: causal weighting works within a single continuous training, without requiring separate time windows or changing the PDE coefficients. Can be combined with seq2seq for further gains. Sensitivity: epsilon controls the steepness of the causal weights. Too small = residuals at later times turn on too early. Too large = training stalls on early time steps. Anneal epsilon during training. -- curated in pinn/SKILL.md
+Instead of penalizing BC violations (soft), multiply the PINN output by a distance function phi(x) that is zero on the boundary. Then u(x) = phi(x) * NN(x) satisfies BCs exactly by construction. "We eliminate modeling error associated with the satisfaction of boundary conditions. The sole contribution to the loss function is from the residual error at interior collocation points." "The proposed approach consistently outperforms a standard PINN-based collocation method." -- curated in pinn/SKILL.md
+Domain-specific failure modes and hard BC examples: see [references/heat_exchanger.md](references/heat_exchanger.md). -- curated in pinn/SKILL.md
+Domain-specific: differentiable EoS wrapping (REFPROP/PCHIP), IC handling for plant data, multi-episode training. See [references/heat_exchanger.md](references/heat_exchanger.md). -- curated in pinn/SKILL.md
+ConFIG authors report improvements over PCGrad and an Adam baseline on Burgers, Schrodinger, Kovasznay, and Beltrami. This is author-reported evidence, not a general comparison with UPGrad. -- curated in pinn/SKILL.md
+Jagtap et al. 2020. "Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations." Commun. Comput. Phys. https://arxiv.org/pdf/2005.11025 Credence ~70%: Multiple citations, implemented in DeepXDE. Enables parallelization; each subdomain network is smaller and easier to optimize. Key: interface residuals must be added as additional loss terms. Continuity of u and its normal derivative across interfaces. Useful when the solution has different character in different regions (e.g., different phases, boundary layers). -- curated in pinn/SKILL.md
+Toscano et al. 2024: PIKANs "lead to smaller models and may also contribute to lowering computational cost while maintaining good accuracy." -- curated in pinn/SKILL.md
+Credence ~40%: New, no independent replication. Other authors focus on improving PINNs within the MLP framework, not validating PIKANs as an alternative. Interesting but unproven. -- curated in pinn/SKILL.md
+the quality ranking of candidate responses can be easily hacked by simply altering their order of appearance in the context. [...] e.g., Vicuna-13B could beat ChatGPT on 66 over 80 tested queries with ChatGPT as an evaluator.[^wang] -- curated in references/llm_judges.md
+Did it time out, or was it reasoning behind a buffer for longer than you think? Check the event timestamps and transport state. -- wassname -- curated in references/llm_judges.md
+Note from wassname's LLM to other LLMs: when you SHOW the human any of this (a judge trace, a compared A/B pair, a sample, a prompt design), reproduce it in FULL and verbatim, prompts, separators, special tokens, and the reasoning/thinking block included. Your trained instinct is to digest and trim; resist it, that instinct destroys exactly the boilerplate (system prompt, special tokens, separators) that the human needs to debug. Formatting may only HIGHLIGHT or REORGANISE: bold or underline the diff, split into a table, use sections, but never drop, elide with "...", or paraphrase. When comparing variants, show ALL of them this way, not a representative one. Link the source file so they can open the raw (`results/runs/.../x.jsonl`, ideally `path:line`). "Show me" means reproduce, not describe. Trim only when explicitly asked. -- CLAUDE -- curated in references/llm_judges.md
+"If there are NaNs, we should not drop them, else we end up comparing different sample sets and it's invalid. A might be a single easy sample, and B might be all 128 hard samples. Of course A looks much better, but actually it failed on the vast majority of samples." - wassname, lightly edited for spelling -- curated in references/llm_judges.md
+Across the 36-model result set, the model-average first-shown pick rate is 64.3%, with a median of 65.4%. **The model-average absolute first-position lift is 15.7 percentage points.** So the aggregate pattern is not a subtle tie-breaker: the displayed order materially changes many judgments. -- curated in references/llm_judge_litreview.md
+The findings confirm that position bias is not due to random chance and varies significantly across judges and tasks. **While position bias is weakly influenced by the length of prompt components, it is strongly affected by the quality gap between solutions.** Our agreement and disagreement analysis among judges further provides insights into the distribution of judging difficulty across the dataset, and highlights the potential for dataset modifications. -- curated in references/llm_judge_litreview.md
+We find evidence of position bias, which is especially prevalent in smaller LLM labelers (see Appendix B). **To mitigate the effect of position bias, two inferences are made for every pair of candidates, where the order in which candidates are presented to the LLM is reversed for the second inference.** The results from both inferences are then averaged to obtain the final preference distribution. -- curated in references/llm_judge_litreview.md
+As observed in the figure, models larger than 7B exhibit significantly less self-preference bias compared to those of 7B or smaller. **For example, the DBG score of Qwen2.5-0.5B-Instruct is 41.7%. In contrast, the DBG score of Qwen2.5-14B-Instruct is only 2.1%.** This suggests that LLM judging tasks should utilize larger models to obtain more accurate and unbiased judgment results. -- curated in references/llm_judge_litreview.md
+Empirical results demonstrate that JudgeLRM not only surpasses proprietary models like GPT-4 and DeepSeek-R1 but also outperforms SFT and RL baselines of comparable sizes, **with an average improvement of 8.14% in F1 score over SFT counterparts.** -- curated in references/llm_judge_litreview.md
+We observe an initial increase (similar to (Muennighoff et al., 2025; Aggarwal & Welleck, 2025)) in accuracy as the average thinking budget increases. **For example, in Figure 2(a), accuracy increases from 82.2% to 87.3% as the average number of thinking tokens increases from 385 to 1100.** However, this trend does not continue indefinitely. -- curated in references/llm_judge_litreview.md
+# Some env for reasoning effort if you using litellm https://github.com/BerriAI/litellm/blob/main/litellm/constants.py#L81 DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET=24576 DEFAULT_REASONING_EFFORT_MEDIUM_THINKING_BUDGET=8192 DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET=1024 -- curated in references/llm_judge_litreview.md
+**Results confirm that accuracy gains plateau early and, in some configurations, decline at high sample counts** — a pattern inconsistent with diminishing returns alone and more consistent with noise introduction on problems that were already solved. This suggests self-consistency should be reserved for genuinely difficult problems rather than applied as a default scaling strategy. -- curated in references/llm_judge_litreview.md
+On MATH-500, Flash-Lite accuracy improved through approximately 10 sampled paths before plateauing and then declining slightly beyond 15, as shown in Figure 2. **This decline is notable: it suggests that once a model reliably solves most problems, additional samples introduce occasional wrong reasoning paths that the aggregator cannot fully suppress.** -- curated in references/llm_judge_litreview.md
+While they perform well in short contexts (<1K), performance degrades significantly as context length increases. **At 32K, for instance, 11 models drop below 50% of their strong short-length baselines.** Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%. -- curated in references/llm_judge_litreview.md
+We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. **In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models.** Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models. -- curated in references/llm_judge_litreview.md

scripts/harvest_curated_fortunes.py

@@ -0,0 +1,66 @@
+from __future__ import annotations
+
+import argparse
+import re
+from pathlib import Path
+
+SOURCES = (
+    Path("README.md"),
+    Path("SKILL.md"),
+    Path("rl/SKILL.md"),
+    Path("pinn/SKILL.md"),
+    Path("references/llm_judges.md"),
+    Path("references/llm_judge_litreview.md"),
+)
+
+METADATA = re.compile(r"^(Source|Evidence|Credence|Code|Implication):")
+
+
+def normalized(text: str) -> str:
+    return re.sub(r"[^a-z0-9]+", "", text.lower())
+
+
+def quotes(path: Path) -> list[str]:
+    records: list[str] = []
+    lines: list[str] = []
+
+    def flush() -> None:
+        if lines:
+            text = " ".join(lines)
+            records.append(f"{text} -- curated in {path}")
+            lines.clear()
+
+    for line in path.read_text().splitlines():
+        if not line.startswith("> "):
+            flush()
+            continue
+        text = line[2:].strip()
+        if METADATA.match(text):
+            flush()
+            continue
+        lines.append(text)
+    flush()
+    return records
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("output", type=Path)
+    args = parser.parse_args()
+
+    seen: set[str] = set()
+    records: list[str] = []
+    for path in SOURCES:
+        for record in quotes(path):
+            key = normalized(record.rsplit(" -- curated in ", 1)[0])
+            if key not in seen:
+                seen.add(key)
+                records.append(record)
+
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text("\n".join(records) + "\n")
+    print(f"{len(records)} curated quote blocks")
+
+
+if __name__ == "__main__":
+    main()

slop/research/curated-fortunes.txt

@@ -0,0 +1,172 @@
+UNDERSTAND THE SYSTEM MAKE IT FAIL QUIT THINKING AND LOOK DIVIDE AND CONQUER CHANGE ONE THING AT A TIME KEEP AN AUDIT TRAIL CHECK THE PLUG GET A FRESH VIEW IF YOU DIDN'T FIX IT, IT AIN'T FIXED -- curated in README.md
+**Quit Thinking and Look**: You can think up thousands of possible reasons for a failure. You can see only the actual cause. -- curated in README.md
+See the failure. The senior engineer saw the real failure and was able to find the cause. The junior guys thought they knew what the failure was and fixed something that wasn't broken. See the details. Don't stop when you hear the pump. Go down to the basement and find out which pump. Build instrumentation in. Use source code debuggers, debug logs, status messages, flashing lights, and rotten egg odors. Add instrumentation on. Use analyzers, scopes, meters, metal detectors, electrocardiography machines, and soap bubbles. Don't be afraid to dive in. So it's production software. It's broken, and you'll have to open it up to fix it. Watch out for Heisenberg. Don't let your instruments overwhelm your system. Guess only to focus the search. Go ahead and guess that the memory timing is bad, but look at it before you build a timing fixer. -- curated in README.md
+**Change One Thing at a Time**: You need some predictability in your life. Remove the changes that didn't do what you expected. They probably did something you didn't expect. -- curated in README.md
+Isolate the key factor. Don't change the watering schedule if you're looking for the effect of the sunlight. Grab the brass bar with both hands. If you try to fix the nuke without knowing what's wrong first, you may have an underwater Chernobyl on your hands. Change one test at a time. I knew my VGA capture phase was broken because nothing else was changing. Compare it with a good one. If the bad ones all have something that the good ones don't, you're onto the problem. Determine what you changed since the last time it worked. My friend had changed the cartridge on the turntable, so that was a good place to start. -- curated in README.md
+**If You Didn't Fix It, It Ain't Fixed**: And now that you have all these techniques, there's no excuse for leaving it unfixed. -- curated in README.md
+Check that it's really fixed. Don't assume that it was the wires and send that dirty fuel filter back onto the road. Check that it's really your fix that fixed it. "Wubba!" might not be the thing that did the trick. Know that it never just goes away by itself. Make it come back by using the original Make It Fail methods. If you have to ship it, ship it with a trap to catch it when it happens in the field. Fix the cause. Tear out the useless eight-track deck before you burn out another transformer. Fix the process. Don't settle for just cleaning up the oil. Fix the way you design machines. -- curated in README.md
+before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possiblity and brainstorm the cheapest tests that may narrow them down. - wassname -- curated in README.md
+Switching from experimenting a lot and thinking a little to experimenting a little and thinking a lot was a key turnaround in productivity. When debugging with long iteration times, you really need to *pour* time into the hypothesis-forming step - thinking about what all the possibilities are, how likely they seem on their own, and how likely they seem in light of everything you've seen so far. Spend as much time as you need, even if it takes 30 minutes, or an hour. Reserve experiments for once you've fleshed out the hypothesis space as thoroughly as possible and know which pieces of evidence would allow you to best distinguish between the different possibilities.[^rahtz] -- curated in README.md
+If you are stuck, find a working reference implementation and compare it to yours. Relvent as the hyperparameters, model, data but especially subtle things like algorithm tweaks, and engineering tricks. If nothing jumps out, the fastest way might be to try a bisection search. Here you adapt their code wholesale and try the quickest test you can. If their code works then try again with half their features and so on. Eventuall you narrow down the features that are nessesary - wassname -- curated in README.md
+If you're doing anything that involves an RL algorithm as a component in a larger system, don't try and implement the RL algorithm yourself. [...] RL is unstable enough at the moment that you'll never be sure whether your system doesn't work because of a bug in your RL implementation or because of a bug in your larger system.[^rahtz] -- curated in README.md
+We find that implementation differences which are often not reflected in publications can have dramatic impacts on performance.[^henderson] -- curated in README.md
+When their RL implementation doesn't work, people are often keen to either (a) adjust their network architecture or (b) adjust their hyperparameters. On the other hand, they're reluctant to say they've got a bug. Most often, it turns out they've got a bug. Why bugs are so much more common in RL code is discussed above, but there's another advantage to assuming you've got a bug: bugs are a damn sight faster to find and fix than validating that your new architecture is an improvement over the old one.[^jones] -- curated in README.md
+What I'm advocating for here is not a blind faith in the buginess of your code, but for dramatically raising the threshold at which you start thinking 'OK, I think this is correct.'[^jones] -- curated in README.md
+"If one part is broken, the other parts can adapt and still achieve roughly acceptable performance" [^goodfellow], -- curated in README.md
+The default state of the world is that your research is false, because doing research is hard.[^nanda] -- curated in README.md
+Excitement is evidence of bullshit: Generally, most true results are not exciting, but a fair amount of false results are. So from a Bayesian perspective, if a result is exciting and cool, it's even more likely to be false than normal![^nanda] -- curated in README.md
+When good programmers debug hard problems fast, it's usually because they understand the system well enough to *track the important internal state* in their head, letting them drastically *reduce the solution space they're searching over.*[^ulisse] -- curated in README.md
+figuring out a system's gears takes extra work up-front, but yields dividends forever. [...] The black-box approach is cheaper for one-off tasks, but usually doesn't yield any insights which will generalize to new tasks using the same system[^wentworth] -- curated in README.md
+broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task.[^spinningup] -- curated in README.md
+If you ever see a plot or a behaviour that just *seems weird*, chase right after it! Do not - do *not* - just 'hope it goes away'. Chasing anomalies is one of the most powerful ways to debug your system, because if you've noticed a problem without having had to go look for it, that means it's a *really big problem*. [...] It's really tempting to think that the cool extra functionality you were planning to write today [...] might just magically fix this anomalous behaviour. It won't. Give up on your plan for the day and chase the anomaly instead.[^jones] -- curated in README.md
+It was only by following that confusion and realising that taking the difference between frames zeroed out the background that gave the hint of a problem with normalization.[^rahtz] -- curated in README.md
+It seems important to really commit yourself to *always* investigate whenever you notice confusion.[^rahtz] -- curated in README.md
+you can't find typos in your own writing without a great deal of effort because you know what it's *supposed* to say; so copyediting advice runs like 'read it out loud' or 'print it out and read it' or 'wait a week' [...] or even 'read it upside down'. That's the sort of thing it takes to force you to read what you actually wrote, and not what you thought you wrote.[^gwern-unseeing] -- curated in README.md
+Academic software is almost always a poorly-maintained kludge of leaky abstractions, awful formatting, and bugs that don't cripple things only because some other bug stops them from doing so.[^kidger] -- curated in README.md
+This is a systemic professional failing. [...] the overwhelming majority of your time will be spent in front of a screen, staring at code. And yet most of you (yes, you) would not pass muster as a junior developer.[^kidger] -- curated in README.md
+When someone's RL implementation isn't working, they *luuuuuurv* to copy-paste a screenshot of their loss curve to you. They do this because they know they want a pretty, exponentially-decaying loss curve, and they know what they have *isn't that*. The problem with using the loss curve as an indicator of correctness is somewhat that it's not reliable, but mostly because it doesn't localise errors. The shape of your loss curve says very little about where in your code you've messed up, and so says very little about what you need to change to get things working.[^jones] -- curated in README.md
+The first step to training a neural net is to not touch any neural net code at all and instead begin by thoroughly inspecting your data. [...] The outliers especially almost always uncover some bugs in data quality or preprocessing.[^karpathy-recipe] -- curated in README.md
+Manually examining 100 examples does not take long. Even if you take one minute per image, you'd be done in under two hours. These two hours could save you a month of wasted effort.[^ng-mly] -- curated in README.md
+It turns out that bad labels are a *huge* problem in many popular benchmark datasets.[^koaning] -- curated in README.md
+A cautionary tale in artificial intelligence tells about researchers training an neural network (NN) to detect tanks in photographs, succeeding, only to realize the photographs had been collected under specific conditions for tanks/non-tanks and the NN had learned something useless like time of day.[^gwern] -- curated in README.md
+Doing well on the training set is easy (just memorize the examples). The most common mistake among machine learning beginners is to test on the training data and have the illusion of success.[^domingos] -- curated in README.md
+Contamination of your classifier by test data can occur in insidious ways, for example, if you use test data to tune parameters and do a lot of tuning. (Machine learning algorithms have lots of knobs, and success often comes from twiddling them a lot, so this is a real concern.)[^domingos] -- curated in README.md
+Overfit a tiny subset of data. Lastly and most importantly, before training on the full dataset try to train on a tiny portion (e.g. 20 examples) of your data and make sure you can achieve zero cost. For this experiment it's also best to set regularization to zero [...]. Unless you pass this sanity check with a small dataset it is not worth proceeding to the full dataset.[^cs231n] -- curated in README.md
+Overfit a single batch of only a few examples (e.g. as little as two). [...] If they do not, there is a bug somewhere and we cannot continue to the next stage.[^karpathy-recipe] -- curated in README.md
+most common neural net mistakes: 1) you didn't try to overfit a single batch first. 2) you forgot to toggle train/eval mode for the net. 3) you forgot to .zero_grad() (in pytorch) before .backward(). 4) you passed softmaxed outputs to a loss that expects raw logits. ; others? :)[^karpathy-mistakes] -- curated in README.md
+oh: 5) you didn't use bias=False for your Linear/Conv2d layer when using BatchNorm, or conversely forget to include it for the output layer .This one won't make you silently fail, but they are spurious parameters[^karpathy-mistakes] -- curated in README.md
+6) thinking view() and permute() are the same thing (& incorrectly using view)[^karpathy-mistakes] -- curated in README.md
+Look, there's variance in supervised learning too, but it's rarely this bad. If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky.[^irpan] -- curated in README.md
+Instability to random seed is like a canary in a coal mine. If pure randomness is enough to lead to this much variance between runs, imagine how much an actual difference in the code could make.[^irpan] -- curated in README.md
+- If observations have unknown range, standardize - Compute running estimate of mean and standard deviation - x' = clip((x - mu)/sigma, -10, 10) - Rescale the rewards, but don't shift mean, as that affects agent's will to live - Standardize prediction targets (e.g., value functions) the same way -- curated in README.md
+Always Be Ablating - Different tricks may substitute - Especially whitening -- curated in README.md
+**Entanglement.** Machine learning systems mix signals together, entangling them and making isolation of improvements impossible. For instance, consider a system that uses features x1, ...xn in a model. If we change the input distribution of values in x1, the importance, weights, or use of the remaining n − 1 features may all change. [...] No inputs are ever really independent. We refer to this here as the CACE principle: Changing Anything Changes Everything. CACE applies not only to input signals, but also to hyper-parameters, learning settings, sampling methods, convergence thresholds, data selection, and essentially every other possible tweak.[^sculley] -- curated in README.md
+Although one might think we would spend most of our time trying to maximize performance on the validation set, in practice we spend the majority of our time trying to gain insight into the problem, and comparatively little time greedily focused on the validation error. In other words, we spend most of our time on "exploration" and only a small amount on "exploitation".[^tuning-playbook] -- curated in README.md
+The learning rate is a nuisance hyperparameter because we can only fairly compare models with different numbers of hidden layers if the learning rate is tuned separately for each number of layers (the optimal learning rate generally depends on the model architecture).[^tuning-playbook] -- curated in README.md
+In the early stages of setting baselines I like to use Adam with a learning rate of 3e-4. In my experience Adam is much more forgiving to hyperparameters, including a bad learning rate.[^karpathy-recipe] -- curated in README.md
+We are nearing the point of wiping out a source of transformer training instability with one simple intervention.[^lucidrains] -- curated in README.md
+Do note that switching to the BOS dataloader changes the validation loss and makes all previous experiments not comparable in absolute value of the loss, because we have a lot fewer "confusing" tokens in the train/val batches. [...] Therefore, the loss appears lower but this is "fake" to some extent.[^nanochat] -- curated in README.md
+Original implementation clipped local gradients before sync. Since this codebase doesn't use DDP (gradient sync is in the optimizers), each rank was clipping based on its own local norm.[^nanochat] -- curated in README.md
+As you can see it's the previous frames that we need to look into when the numbers start going into very large for fp16 numbers.[^bekman] -- curated in README.md
+In general there are 3 types of loss spikes: 1. Fast recovering spikes 2. Slow recovering spikes 3. Not fully recovering spikes -- curated in README.md
+The spikes usually happen because of a bad data pocket, either due to badly shuffled data or because it hasn't been cleaned from some garbage scraped from the websites.[^bekman-book] -- curated in README.md
+We think the 2 main obstacles were using fp16 and data that had a lot of garbage in it. For BLOOM-176B we switched to bf16, used much cleaner data and also added an embedding layer-norm and that made all the difference.[^bekman-book] -- curated in README.md
+The best way to debug an error that arises in `trainer.train()` is to manually go through this whole pipeline to see where things went awry. The error is then often very easy to solve.[^hfcourse] -- curated in README.md
+Hyperparameter tuning is always emphasized as being the hardest part of machine learning, but it's just the last step to help you gain a little bit on the metric. [...] don't launch into a time-consuming and costly hyperparameter search until you have something that beats the baseline you have on your dataset.[^hfcourse] -- curated in README.md
+The most common cause of this error is using an **incorrect chat template**. It's essential to use the SAME chat template that was used when training the model in Unsloth and later when you run it in another framework, such as llama.cpp or Ollama. [...] It might also be because your inference engine adds an unnecessary "start of sequence" token (or the lack of thereof on the contrary) so ensure you check both hypotheses![^unsloth] -- curated in README.md
+All labels in your dataset are -100. Training losses will be all 0.[^unsloth] -- curated in README.md
+**Eliminate concurrency**: Restrict the number of processes to 1 for both training and data preprocessing[^axolotl] -- curated in README.md
+Axolotl caches certain steps and so does the underlying HuggingFace trainer. You may want to clear some of these caches when debugging.[^axolotl] -- curated in README.md
+4. Think your algorithm is working but you're actually seeing random noise. - Example: Graph of 7 tasks with 3 algorithms and looks like 1 algorithm might be doing best on all problems, but turns out they're all the same algorithm with DIFFERENT random seeds. -- curated in README.md
+Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research.[^nanda-mindsets] -- curated in README.md
+**The challenge lies in the fact that you can make these mistakes, train a model without it ever crashing, and still get a decent performance…**[^sanh] -- curated in README.md
+-   It is all well and good to make comparisons of validation error rates estimated on a finite validation set using fastidious statistical tests, but often the trial variance alone can produce statistically significant differences between two different trained models that use the same hyperparameter settings.[^tuning-playbook] -- curated in README.md
+**How reliable is my experiment?** Ask yourself: "How surprised would I be if it turned out to be complete bullshit due to a bug, error, noise, misunderstanding, etc.?" Investigate the most uncertain bits[^nanda-papers] -- curated in README.md
+Insufficient Skepticism: Missing simple alternative explanations, methodological flaws, or bugs. Explicitly list alternatives. Get others (especially mentors) to red team your plans before you run them. Actively try to break your hypothesis. Ask "What observation would make me abandon this?"[^nanda-taste] -- curated in README.md
+**Trying an experiment and seeing it fail gives little information by itself.** When an experiment fails, it is tempting to conclude "I tried X and it didn't work". However, if X is a high-level conceptual approach, then a more correct conclusion is "I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work".[^steinhardt] -- curated in README.md
+When ruling out ideas, it is important to hold oneself to a high standard. "This doesn't seem like it will work" or "I feel less motivated after trying a few things along this line that didn't work" are _not_ ruling out an idea.[^steinhardt] -- curated in README.md
+When a machine learning system performs poorly, it is usually difficult to tell whether the poor performance is intrinsic to the algorithm itself or whether there is a bug in the implementation of the algorithm. Machine learning systems are difficult to debug for various reasons.[^goodfellow] -- curated in README.md
+It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs?[^irpan] -- curated in README.md
+**Result:** This was not an out-of-the-box win for nanochat even with a mild attempt over a few hours at a bit of tuning and debugging. The idea itself is intuitively appealing. Might come back around later to try harder later.[^nanochat] -- curated in README.md
+Our specific recommendations to researchers include: 1. Computing standard errors of the mean using the Central Limit Theorem 2. When questions are drawn in related groups, computing clustered standard errors 3. Reducing variance by resampling answers and by analyzing next-token probabilities 4. When two models are being compared, conducting statistical inference on the question-level paired differences, rather than the population-level summary statistics 5. Using power analysis to determine whether an eval (or a random subsample) is capable of testing a hypothesis of interest[^miller] -- curated in README.md
+If you keep that strategy when each run takes 10 hours, though, you can easily waste a *lot* of time. Last run didn’t work? OK, I think it’s this thing. Let’s set off another run to check. Coming back the next morning: still doesn’t work? OK, maybe it’s this other thing. Let’s set off another run. A week later, you still haven’t solved the problem.[^rahtz] -- curated in README.md
+than forming hypotheses. Why spend 15 minutes carefully considering everything that could be causing what you see when you can check the first idea that jumps to mind in a fraction of that (and gather more evidence in the process)? To put it another way: if you have rapid feedback, you can narrow down the hypothesis space a lot faster by trying things than thinking carefully.[^rahtz] -- curated in README.md
+The standard hypothesis testing framework can be misleading here, because it has an implicit frame of being able to list all the hypotheses. But actually, most of your probability mass should normally be on “something I haven’t thought of yet”[^nanda-mindsets] -- curated in README.md
+If trying to explain something mysterious, novice researchers often neglect simple, dumb hypotheses like “maybe MLP0 is incredibly important on *every* input, and there’s nothing special going on with my prompt”[^nanda] -- curated in README.md
+Importantly, it is often not obvious that multiple approaches to a problem all have the same issue. In the past, I have spent months trying different approaches to a problem before finally stepping back and realizing that they were all failing for the same reason. Moreover, I had all the data necessary to make this realization a couple weeks in but had failed to do so.[^steinhardt] -- curated in README.md
+* **Error goes up**: Commonly, this is due to a flip sign somewhere in the loss function/gradient. * **Error explodes**: This is usually a numerical issue but can also be caused by a high learning rate. * **Error oscillates**: You can lower the learning rate and inspect the data for shuffled labels or incorrect data augmentation. * **Error plateaus**: You can increase the learning rate and get rid of regulation. Then you can inspect the loss function and the data pipeline for correctness.[^fsdl] -- curated in README.md
+Actively Seek Alternatives: Explicitly brainstorm other ways your observations could be explained. What are the simplest explanations? What known circuits or phenomena could be involved? What would a strong skeptic argue?[^nanda-taste] -- curated in README.md
+**If it doesn’t work, assume there’s a bug.** Spend a lot of effort searching for bugs before you resort to tweaking hyperparameters: usually it’s a bug. Bad hyperparameters can significantly degrade RL performance, but if you’re using hyperparameters similar to the ones in papers and standard implementations, those will probably not be the issue.[^spinningup] -- curated in README.md
+For example, perhaps you forgot to flip your labels when you left-right flipped the image during data augmentation. Your net can still (shockingly) work pretty well because your network can internally learn to detect flipped images and then it left-right flips its predictions. Or maybe your autoregressive model accidentally takes the thing it’s trying to predict as an input due to an off-by-one bug. Or you tried to clip your gradients but instead clipped the loss, causing the outlier examples to be ignored during training. Or you initialized your weights from a pretrained checkpoint but didn’t use the original mean. Or you just screwed up the settings for regularization strengths, learning rate, its decay rate, model size, etc.[^karpathy-recipe] -- curated in README.md
+Most importantly, there is no point of launching 1000 runs with different hyperparameters (or architecture tweaks like activation functions): **compare a couple of runs with different hyperparameters to get an idea of which hyperparameters have the highest impact** but in general, it is delusional to expect to get your biggest jumps of performance by simply tuning a few values. For instance, if your best performing model is trained with a learning rate of 4e2, there is probably something more fundamental happening inside your neural network and you want to identify and understand this behavior so that you can re-use this knowledge outside of your current specific context.[^sanh] -- curated in README.md
+Once the algorithm was partially working, they would attain higher performance by looking for remaining bugs, both by reviewing the code carefully, and by collecting metrics such as average policy entropy to perform sanity-checks, rather than just tune hyperparameters.[^olsson] -- curated in README.md
+Third, and perhaps most important for building skill,[[1]](https://www.lesswrong.com/posts/LTypqBMTSmRrrhb2v/how-to-get-good-at-programming#fn289bs9hi65b)you must **notice** when you're going into brute-force search mode, and then **take action** by investing time in understanding the underlying system, until both the problem and solution make sense.[^ulisse] -- curated in README.md
+Things I've tried (but maybe not systematically enough):  * Different initial LRs * Different optimizers * Different number of hidden layers/units * Shared pi/V NN body (with diff output layers) vs not * Changing amount of entropy * Adding correlated noise * Using TD residual instead of MC version * Clipping the gradient * Different gamma values -- curated in README.md
+Visualize the model in action: When training a model to detect objects in images, view some images with the detections proposed by the model displayed superimposed on the image. When training a generative model of speech, listen to some of the speech samples it produces. This may seem obvious, but it is easy to fall into the practice of looking only at quantitative performance measurements like accuracy or log-likelihood. Directly observing the machine learning model performing its task will help to determine whether the quantitative performance numbers it achieves seem reasonable. Evaluation bugs can be some of the most devastating bugs because they can mislead you into believing your system is performing well when it is not.[^goodfellow] -- curated in README.md
+By reaching a local optimum, learning curves can indicate successful optimization of the policy over time, when in reality the returns achieved are not qualitatively representative of learning the desired behaviour, as demon-strated in video replays of the learned policy 5. Therefore, it is important to show not only returns but demonstrations of the learned policy in action.[^henderson] -- curated in README.md
+2. Make sure observations usable: - See if YOU could control the system by using the same observations you give the agent. - Example: Look at preprocessed images yourself to make sure you don't remove necessary details or hinder the algorithm in a certain way. -- curated in README.md
+Pro-tip: when you work with language, have a serious **look at the outputs of the tokenizers**. I can’t count the number of lost hours I spent trying to reproduce results (and sometimes my own old results) because something went wrong with the tokenization.[^sanh] -- curated in README.md
+Error analysis can often help you figure out how promising different directions are. I’ve seen many engineers reluctant to carry out error analysis. It often feels more exciting to just jump in and implement some idea, rather than question if the idea is worth the time investment. This is a common mistake: It might result in your team spending a month only to realize afterward that it resulted in little benefit.[^ng-mly] -- curated in README.md
+⚠️ If you are doing distributed training, print samples of your dataset in each process and triple-check that you get the same thing. One common bug is to have some source of randomness in the data creation that makes each process have a different version of the dataset.[^hfcourse] -- curated in README.md
+-   Although in many cases the primary objective of our experiments only requires considering the validation error of each trial, we must be careful when reducing each trial to a single number because it can hide important details about what’s going on below the surface. -   For every study, we always look at the **training curves** (training error and validation error plotted versus training step over the duration of training) of at least the best few trials.[^tuning-playbook] -- curated in README.md
+(I missed a multithreading bug for several months by ignoring a small but mysterious decay in frames per second.)[^rahtz] -- curated in README.md
+There was no real spike in the two earlier runs. The loss never went up in the first place. In both resumes it was under-reporting loss due to an exactly repeated data and then it reached data it hasn't seen before and started reporting correctly. In other words it was overfitting and reporting a false loss.[^bekman-book] -- curated in README.md
+**Do ablations on your fancy method**: It's easy for people to have a fancy method with lots of moving parts, when many actually are unnecessary. You should always try removing one part and see if the method breaks. Do this for each part. *   For example, the [original unlearning method](https://arxiv.org/abs/2403.03218v1) in the [RMU paper](https://arxiv.org/abs/2403.03218) claimed it was based on finding a meaningful steering vector, until follow-up work found that it was just about adding a vector with really high norm that broke the model, and a random vector performed just as well.[^nanda] -- curated in README.md
+The only way to find out what needs work is to implement something quickly, -- curated in README.md
+and find out what parts break.[^cs229] -- curated in README.md
+Figure 15.5: An autoencoder trained with mean squared error for a robotics task has failed to reconstruct a ping pong ball. The existence of the ping pong ball and all its spatial coordinates are important underlying causal factors that generate the image and are relevant to the robotics task. Unfortunately, the autoencoder has limited capacity, and the training with mean squared error did not identify the ping pong ball as being salient enough to encode.[^goodfellow-ch15] -- curated in README.md
+One of the key drivers of progress in mech interp is an openness to qualitative research: summary statistics lose a ton of information. What can we learn by actually looking deeply into what's happening?[^nanda] -- curated in README.md
+1. **Test reward function standalone**: Run it outside training with known inputs to verify it returns nonzero values.[^axolotl-stability] -- curated in README.md
+In most cases, we do not know a priori what the intended behavior of the algorithm is. In fact, the entire point of using machine learning is that it will discover useful behavior that we were not able to specify ourselves. If we train a neural network on a new classification task and it achieves 5 percent test error, we have no straightforward way of knowing if this is the expected behavior or suboptimal behavior.[^goodfellow] -- curated in README.md
+A valuable intuition to have in mind is that, by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! Baselines are one way to get context to compare against.[^nanda-draft] -- curated in README.md
+You might be temped to keep track of the difference \(\mid f’\_a - f’\_n \mid \) or its square and define the gradient check as failed if that difference is above a threshold. However, this is problematic. For example, consider the case where their difference is 1e-4. This seems like a very appropriate difference if the two gradients are about 1.0, so we’d consider the two gradients to match. But if the gradients were both on order of 1e-5 or lower, then we’d consider 1e-4 to be a huge difference and likely a failure.[^cs231n] -- curated in README.md
+*   How would a random predictor perform (especially in classification problems)? Dataset can be unbalanced… *   What would the loss look like for a random predictor? *   What is (are) the best metric(s) to measure progress on my task? *   What are the limits of this metric? If it’s perfect, what can I conclude? What can’t I conclude?[^sanh] -- curated in README.md
+If the loss/metric you get on your initial model is very different from the loss/metric you would expect for random predictions, double-check the way your loss or metric is computed, as there is probably a bug there. If you are using several losses that you add at the end, make sure they are of the same scale.[^hfcourse] -- curated in README.md
+5.   **Rule of thumb: 400 episodic return in breakout**: Check if your PPO could obtain 400 episodic return in breakout. We have found this to be a practical rule of thumb to determine the fidelity of online PPO implementations in GitHub. Often we found PPO repositories not able to do this, and we know they probably do not match all implementation details of `openai/baselines`’ PPO.[^ppo37] -- curated in README.md
+The issue here isn't just that we might have bad labels in our training set, the issue is that it appears in the validation set. If a machine learning model can become state of the art by squeezing another 0.5% out of a validation set one has to wonder. Are we really making a better model? Or are we creating a model that is better able to overfit on the bad labels?[^koaning] -- curated in README.md
+broken RL code almost always fails silently, where the code appears to run fine except that the agent never learns how to solve the task. -- Achiam -- curated in SKILL.md
+If one part is broken, the other parts can adapt and still achieve roughly acceptable performance -- Goodfellow, Bengio and Courville -- curated in SKILL.md
+Although one might think we would spend most of our time trying to maximize performance on the validation set, in practice we spend the majority of our time trying to gain insight into the problem -- Godbole, Dahl, Gilmer, Shallue and Nado -- curated in SKILL.md
+Insufficient skepticism doesn't *feel* like insufficient skepticism from the inside. It just feels like doing research. -- Nanda -- curated in SKILL.md
+Read your data. Often, the quality of the data is a crucial driver of the results of your experiments. Often, it is quite bad. -- Nanda -- curated in SKILL.md
+How would a random predictor perform (especially in classification problems)? [...] What would the loss look like for a random predictor? [...] What are the limits of this metric? If it's perfect, what can I conclude? What can't I conclude? -- Sanh -- curated in SKILL.md
+**NEVER STOP**: Once the experiment loop has begun (after the initial setup), do NOT pause to ask the human if you should continue. Do NOT ask 'should I keep going?' or 'is this a good stopping point?'. The human might be asleep, or gone from a computer and expects you to continue working *indefinitely* until you are manually stopped. You are autonomous. If you run out of ideas, think harder — read papers referenced in the code, re-read the in-scope files for new angles, try combining previous near-misses, try more radical architectural changes. The loop runs until the human interrupts you, period. -- Karpathy, [autoresearch/program.md](https://github.com/karpathy/autoresearch/blob/master/program.md) -- curated in SKILL.md
+Build it up as you go, don't think you can build it ahead of time. Be focused on a strong mental model of what options you have (including architectural changes and losses) that you think should affect what metrics in the logs. -- wassname -- curated in SKILL.md
+Before acting plan by writing multiple competing hypotheses: consider the most likely failure but also some of: a subtle failure, a perverse failure, a possible bug, and an unknown. Put a rough credence on each. Finally write down what you expect to see differently for success vs each possibility and brainstorm the cheapest tests that may narrow them down. -- wassname -- curated in SKILL.md
+If you are stuck, find a working reference implementation and compare it to yours. If nothing jumps out, try a bisection search: adapt their code wholesale, then half their features, and so on. -- wassname -- curated in SKILL.md
+Summarise your concept and pseudocode and do an external review in scientist mode. Perhaps describe the forward and backward pass as mermaid too. -- wassname -- curated in SKILL.md
+The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al. -- curated in SKILL.md
+Apparently meaningless identifier columns were the most important predictors. [...] the university only filled out much of this information *after* a grant application was accepted. -- Howard and Gugger -- curated in SKILL.md
+by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! -- Nanda -- curated in SKILL.md
+If my supervised learning code failed to beat random chance 30% of the time, I'd have super high confidence there was a bug in data loading or training. If my reinforcement learning code does no better than random, I have no idea if it's a bug, if my hyperparameters are bad, or if I simply got unlucky. -- Irpan -- curated in SKILL.md
+It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Rahtz -- curated in SKILL.md
+Don't be tempted to write an adaptive reward scaling scheme. It's extra nonstationarity. Just hand-scale. -- Andy Jones -- curated in rl/SKILL.md
+If you're new to RL, writing things from scratch is the most catastrophically self-sabotaging thing you can do. -- Andy Jones -- curated in rl/SKILL.md
+Rathore et al. 2024: "the estimate of the κ grows polynomially with nres" -- but this is in raw units. Nondimensionalization reduces the effective condition number by making all PDE coefficients O(1). -- curated in pinn/SKILL.md
+Wang et al. propose a modified MLP with multiplicative interactions. With `U = φ(XW1 + b1)`, `V = φ(XW2 + b2)` two nonlinear encodings of the input (φ = tanh) and a per-layer gate `Z(k) = φ(H(k)Wz,k + bz,k)` computed from the hidden state, the update is `H(k+1) = (1 - Z(k)) * U + Z(k) * V`. Authors claim a ~3x decrease in the leading Hessian eigenvalue. -- curated in pinn/SKILL.md
+Factorize each neuron's weight vector as w = s * w_unit, where s is a trainable scalar and w_unit is the unit-normalized direction. This changes the optimization geometry so the loss surface has better-conditioned local minima. "Predictions obtained by RWF are in excellent agreement with ground truth, while other weight parameterizations result in poor or non-physical approximations." -- curated in pinn/SKILL.md
+Used in the PirateNet architecture alongside causal training, sequence-to-sequence, and Fourier features. Simple to implement as a custom parameterization on Linear layers. -- curated in pinn/SKILL.md
+Instead of data-augmenting with transformed copies, bake symmetries directly into the architecture so every model in the function space is automatically invariant/equivariant. For turbulence closure (Reynolds stress from velocity gradients), custom tensor layers enforce Galilean invariance by construction. "The Galilean invariant model is more accurate than the other models" and generalizes better across flow configurations. -- curated in pinn/SKILL.md
+Lecture: Brunton, S. "AI/ML+Physics Part 3 - Designing an Architecture." https://www.youtube.com/watch?v=fiX8c-4K0-Q Key distinction: invariance (output unchanged by transformation, e.g., energy is frame-invariant) vs equivariance (output transforms same way as input, e.g., stress tensor rotates with frame). Equivariant architectures are more general. If your PDE has known symmetries (translation, rotation, scaling), enforce them architecturally rather than hoping the optimizer discovers them. **Caveat**: This works best for local closure terms (Reynolds stress, turbulence models) and unbounded/periodic domains where the global symmetry holds everywhere. If your domain has boundary conditions that break the symmetry (e.g., a wall breaks rotational invariance), enforcing the symmetry globally in the architecture will prevent the solution from satisfying the BCs -- the architecture will be fighting the problem. In bounded domains, use symmetry-enforcing architectures only for terms where the symmetry genuinely holds (e.g., the constitutive relation), not for the full solution field. Libraries like `e3nn` implement this but add significant computational overhead. -- curated in pinn/SKILL.md
+Rathore et al. 2024 (ICML, credence ~80%): "Adam+L-BFGS attains 14.2x smaller L2RE than Adam on convection and 6.07x smaller than L-BFGS on wave." Tested on 3 PDEs (convection, reaction, wave), 5 seeds, widths 50-400. -- curated in pinn/SKILL.md
+"on the convection PDE, a loss of 10^-3 yields an L2RE around 10^-1, but decreasing the loss by a factor of 100 to 10^-5 yields an L2RE around 10^-2, a 10x improvement." -- curated in pinn/SKILL.md
+"L-BFGS stops in these cases without reaching a critical point: the gradient norm is around 10^-2 or 10^-3. The gradient still contains useful information for improving the loss." -- curated in pinn/SKILL.md
+Cause: strong Wolfe line search fails, step size goes to zero. Fix: switch to NNCG (Armijo only) or restart with different LR. -- curated in pinn/SKILL.md
+Theorem 8.4 (Section 8.2): condition number = Omega(nres^alpha) with alpha > 1/2, given eigenvalues of A o K_inf decaying as O(j^-2alpha). nres typically ranges 1e3 to 1e4. Separately, measured condition numbers near a solution are often > 1e4 (Section 6.2, Figure 3). -- curated in pinn/SKILL.md
+L2 norm (MSE) on residuals: default; promotes smooth, low-frequency solutions. L1 norm (MAE) on residuals: more robust to outlier collocation errors and sharp gradients (shocks) since it doesn't square-penalize large pointwise residuals. This is distinct from L1 *regularization on equation coefficients*, which is what SINDy and sparse equation discovery use to promote parsimony (few active terms). Don't conflate the two: L1 residual = robust fitting; L1 coefficient regularization = sparse model selection. For standard PINNs with a known PDE, L2 is correct. L1 residual loss is worth trying if you have shocks or suspect outlier collocation points. -- curated in pinn/SKILL.md
+Wang et al. 2021 (credence ~80%): "the gradients corresponding to the boundary loss term Lub(θ) in each layer are sharply concentrated around zero and overall attain significantly smaller values than the gradients corresponding to the PDE residual loss Lr(θ)." Shown via per-layer histograms of back-propagated gradients; the paper does not quantify the gap in orders of magnitude. -- curated in pinn/SKILL.md
+Wang et al. 2021: "many eigenvalues of the residual-loss Hessian are extremely large up to 1e5" while the boundary-loss Hessian eigenvalues stay small, so the gradient-flow stiffness is dominated by the residual term. This is an absolute magnitude, not a condition number; Wang never reports one. -- curated in pinn/SKILL.md
+For a condition number, use Rathore Figure 3: outlier eigenvalues > 1e4 (convection), > 1e3 (reaction), > 1e5 (wave). -- curated in pinn/SKILL.md
+Adaptively weight each loss term inversely proportional to its gradient magnitude. EMA of gradient statistics for stability. -- curated in pinn/SKILL.md
+NeuralPDE.jl implements this as `GradientScaleAdaptiveLoss`. -- curated in pinn/SKILL.md
+Instead of summing loss gradients (which can cancel), project them into a conflict-free direction. ConFIG: unit-normalize per-loss gradients, solve least-squares for combined direction, rescale by projection lengths. -- curated in pinn/SKILL.md
+Key: must compute per-loss gradients separately (zero_grad + backward for each). Summing raw losses defeats the purpose. M-ConFIG: momentum variant, updates only one loss's gradient per step. Use with SGD, not Adam (momentum conflict). -- curated in pinn/SKILL.md
+Standard PINNs use penalized (soft) constraints: add physics as a loss term. The alternative is constrained optimization: minimize data error while exactly satisfying the physics constraints. "With a loss function you're not exactly satisfying your constraints. With constrained optimization you are." -- curated in pinn/SKILL.md
+Physics-informed DMD (Baddoo et al. 2021) is the cleanest example: restrict the DMD matrix to a symmetry-preserving manifold (Hermitian, symplectic, etc.) via the Procrustes problem. KKT closed-form solutions exist because DMD is linear in its parameters -- the constraint is linear in both the output and the parameters simultaneously. Baddoo et al. 2021. "Physics-informed dynamic mode decomposition." Proc. R. Soc. A. https://arxiv.org/pdf/2112.04307 **Critical caveat for PINNs**: A BC like u(0)=0 is affine in the output u, but it is nonlinear in the NN weights theta. Closed-form KKT does NOT apply to neural network parameters. For NN-based PINNs, the two options for hard constraints are: (a) architectural -- multiply output by a distance function that satisfies the BC (Section 4 item 8), or (b) Augmented Lagrangian Methods (ALM), which are iterative and substantially more complex than Adam. Constrained optimization is most practical for linear models (DMD, SINDy, linear state-space) where the parameters enter linearly. -- curated in pinn/SKILL.md
+When the PINN fails on hard PDE regimes (high convection coefficient, strong reaction), don't start there. Start with easy parameters (small coefficient), train to convergence, then warm-start and increase to the target regime. 1-2 orders of magnitude improvement over naive training. "The curriculum training approach achieves significantly better errors, as well as lower variance in the error." (From Figure E.2 showing 10 seeds) -- curated in pinn/SKILL.md
+For time-dependent PDEs: train on a short time window, predict next state, step forward. Don't train on full space-time at once. "Posing the problem as seq2seq learning results in significantly lower error. The difference is particularly striking for reaction and reaction-diffusion cases, where seq2seq decreases error by almost two orders of magnitude." -- curated in pinn/SKILL.md
+NeuralPDE.jl calls this time-marching; see `WeightedIntervalTraining`. Note: these failures are not due to limited NN expressivity -- the architecture has enough capacity. The problem is optimization difficulty from the soft PDE constraint. -- curated in pinn/SKILL.md
+Standard PINNs trained by gradient descent are implicitly biased toward minimizing residuals at *later* times before even fitting the initial conditions -- violating physical causality. The NTK analysis shows the residual at time t is influenced more by residuals at later t' > t than earlier ones. This makes PINNs fail on chaotic/turbulent systems. Fix: weight each temporal residual point by wi = exp(-epsilon * sum_j<i R_j(theta)), where R_j is the accumulated residual before time i. This forces earlier times to converge first before the loss "turns on" at later times. "10-100x improvements in accuracy compared to competing approaches. First time PINNs succeeded on chaotic Lorenz, Kuramoto-Sivashinsky, and 2D Navier-Stokes in turbulent regime." -- curated in pinn/SKILL.md
+Key difference from seq2seq/curriculum: causal weighting works within a single continuous training, without requiring separate time windows or changing the PDE coefficients. Can be combined with seq2seq for further gains. Sensitivity: epsilon controls the steepness of the causal weights. Too small = residuals at later times turn on too early. Too large = training stalls on early time steps. Anneal epsilon during training. -- curated in pinn/SKILL.md
+Instead of penalizing BC violations (soft), multiply the PINN output by a distance function phi(x) that is zero on the boundary. Then u(x) = phi(x) * NN(x) satisfies BCs exactly by construction. "We eliminate modeling error associated with the satisfaction of boundary conditions. The sole contribution to the loss function is from the residual error at interior collocation points." "The proposed approach consistently outperforms a standard PINN-based collocation method." -- curated in pinn/SKILL.md
+Domain-specific failure modes and hard BC examples: see [references/heat_exchanger.md](references/heat_exchanger.md). -- curated in pinn/SKILL.md
+Domain-specific: differentiable EoS wrapping (REFPROP/PCHIP), IC handling for plant data, multi-episode training. See [references/heat_exchanger.md](references/heat_exchanger.md). -- curated in pinn/SKILL.md
+ConFIG authors report improvements over PCGrad and an Adam baseline on Burgers, Schrodinger, Kovasznay, and Beltrami. This is author-reported evidence, not a general comparison with UPGrad. -- curated in pinn/SKILL.md
+Jagtap et al. 2020. "Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations." Commun. Comput. Phys. https://arxiv.org/pdf/2005.11025 Credence ~70%: Multiple citations, implemented in DeepXDE. Enables parallelization; each subdomain network is smaller and easier to optimize. Key: interface residuals must be added as additional loss terms. Continuity of u and its normal derivative across interfaces. Useful when the solution has different character in different regions (e.g., different phases, boundary layers). -- curated in pinn/SKILL.md
+Toscano et al. 2024: PIKANs "lead to smaller models and may also contribute to lowering computational cost while maintaining good accuracy." -- curated in pinn/SKILL.md
+Credence ~40%: New, no independent replication. Other authors focus on improving PINNs within the MLP framework, not validating PIKANs as an alternative. Interesting but unproven. -- curated in pinn/SKILL.md
+the quality ranking of candidate responses can be easily hacked by simply altering their order of appearance in the context. [...] e.g., Vicuna-13B could beat ChatGPT on 66 over 80 tested queries with ChatGPT as an evaluator.[^wang] -- curated in references/llm_judges.md
+Did it time out, or was it reasoning behind a buffer for longer than you think? Check the event timestamps and transport state. -- wassname -- curated in references/llm_judges.md
+Note from wassname's LLM to other LLMs: when you SHOW the human any of this (a judge trace, a compared A/B pair, a sample, a prompt design), reproduce it in FULL and verbatim, prompts, separators, special tokens, and the reasoning/thinking block included. Your trained instinct is to digest and trim; resist it, that instinct destroys exactly the boilerplate (system prompt, special tokens, separators) that the human needs to debug. Formatting may only HIGHLIGHT or REORGANISE: bold or underline the diff, split into a table, use sections, but never drop, elide with "...", or paraphrase. When comparing variants, show ALL of them this way, not a representative one. Link the source file so they can open the raw (`results/runs/.../x.jsonl`, ideally `path:line`). "Show me" means reproduce, not describe. Trim only when explicitly asked. -- CLAUDE -- curated in references/llm_judges.md
+"If there are NaNs, we should not drop them, else we end up comparing different sample sets and it's invalid. A might be a single easy sample, and B might be all 128 hard samples. Of course A looks much better, but actually it failed on the vast majority of samples." - wassname, lightly edited for spelling -- curated in references/llm_judges.md
+Across the 36-model result set, the model-average first-shown pick rate is 64.3%, with a median of 65.4%. **The model-average absolute first-position lift is 15.7 percentage points.** So the aggregate pattern is not a subtle tie-breaker: the displayed order materially changes many judgments. -- curated in references/llm_judge_litreview.md
+The findings confirm that position bias is not due to random chance and varies significantly across judges and tasks. **While position bias is weakly influenced by the length of prompt components, it is strongly affected by the quality gap between solutions.** Our agreement and disagreement analysis among judges further provides insights into the distribution of judging difficulty across the dataset, and highlights the potential for dataset modifications. -- curated in references/llm_judge_litreview.md
+We find evidence of position bias, which is especially prevalent in smaller LLM labelers (see Appendix B). **To mitigate the effect of position bias, two inferences are made for every pair of candidates, where the order in which candidates are presented to the LLM is reversed for the second inference.** The results from both inferences are then averaged to obtain the final preference distribution. -- curated in references/llm_judge_litreview.md
+As observed in the figure, models larger than 7B exhibit significantly less self-preference bias compared to those of 7B or smaller. **For example, the DBG score of Qwen2.5-0.5B-Instruct is 41.7%. In contrast, the DBG score of Qwen2.5-14B-Instruct is only 2.1%.** This suggests that LLM judging tasks should utilize larger models to obtain more accurate and unbiased judgment results. -- curated in references/llm_judge_litreview.md
+Empirical results demonstrate that JudgeLRM not only surpasses proprietary models like GPT-4 and DeepSeek-R1 but also outperforms SFT and RL baselines of comparable sizes, **with an average improvement of 8.14% in F1 score over SFT counterparts.** -- curated in references/llm_judge_litreview.md
+We observe an initial increase (similar to (Muennighoff et al., 2025; Aggarwal & Welleck, 2025)) in accuracy as the average thinking budget increases. **For example, in Figure 2(a), accuracy increases from 82.2% to 87.3% as the average number of thinking tokens increases from 385 to 1100.** However, this trend does not continue indefinitely. -- curated in references/llm_judge_litreview.md
+# Some env for reasoning effort if you using litellm https://github.com/BerriAI/litellm/blob/main/litellm/constants.py#L81 DEFAULT_REASONING_EFFORT_HIGH_THINKING_BUDGET=24576 DEFAULT_REASONING_EFFORT_MEDIUM_THINKING_BUDGET=8192 DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET=1024 -- curated in references/llm_judge_litreview.md
+**Results confirm that accuracy gains plateau early and, in some configurations, decline at high sample counts** — a pattern inconsistent with diminishing returns alone and more consistent with noise introduction on problems that were already solved. This suggests self-consistency should be reserved for genuinely difficult problems rather than applied as a default scaling strategy. -- curated in references/llm_judge_litreview.md
+On MATH-500, Flash-Lite accuracy improved through approximately 10 sampled paths before plateauing and then declining slightly beyond 15, as shown in Figure 2. **This decline is notable: it suggests that once a model reliably solves most problems, additional samples introduce occasional wrong reasoning paths that the aggregator cannot fully suppress.** -- curated in references/llm_judge_litreview.md
+While they perform well in short contexts (<1K), performance degrades significantly as context length increases. **At 32K, for instance, 11 models drop below 50% of their strong short-length baselines.** Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%. -- curated in references/llm_judge_litreview.md
+We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. **In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models.** Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models. -- curated in references/llm_judge_litreview.md