mirror of
https://github.com/wassname/ml_debug.git
synced 2026-06-27 01:00:14 +08:00
wassname
ee4e9a5caa
Gather debugging folklore from more practitioners, each a verbatim quote
checked against a cached source copy (footnoted with line numbers):
- koaning (Vincent Warmerdam), "Bad Labels": benchmark labels are often wrong;
find them with confidence-sorted errors.
- gwern, the tank-detection legend: the canonical data-leakage parable, plus
the scout-mindset twist that it's a likely-unsourced urban legend.
- Patrick Kidger, "Just Know Stuff": why research code is buggy ("kludge ...
bugs that don't cripple things only because some other bug stops them") and
"never accept the kludge". Plus a one-line jaxtyping pointer for shape bugs.
- nanochat (Karpathy): BOS-alignment fake metric improvement; all-ranks must
clip on inf (a multi-GPU bug single-GPU testing hides).
- cleanrl "37 Implementation Details of PPO" -> RL sub-skill, as the canonical
proof that reference-impl details (not ideas) decide whether PPO works.
Trim the lucidrains item to one quote (it had ballooned). Add wassname credit
+ companion-gist link. All 20 footnotes resolve.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
18 lines
1.1 KiB
Markdown
18 lines
1.1 KiB
Markdown
# Just Know Stuff (how to achieve success in an ML PhD) — Patrick Kidger
|
|
|
|
Source: https://kidger.site/thoughts/just-know-stuff/ (2023-01-26). Cached excerpt from the "Software development" section, verbatim.
|
|
|
|
---
|
|
|
|
> Academic software is almost always a poorly-maintained kludge of leaky abstractions, awful formatting, and bugs that don't cripple things only because some other bug stops them from doing so.
|
|
|
|
> This is a systemic professional failing. As an (applied) ML researcher, the overwhelming majority of your time will be spent in front of a screen, staring at code. And yet most of you (yes, you) would not pass muster as a junior developer.
|
|
|
|
> So, how to improve? First of all, never accept the kludge.
|
|
|
|
> You've messed up your Git repo? Figure out the commands to fix it... don't just delete it and clone from the remote.
|
|
|
|
> Focus on writing clean code, based around orthogonal abstractions. When the code starts getting messy - and it will - be willing to refactor your code into something more legible. Avoid both spaghetti code and ravioli code.
|
|
|
|
> When the documentation is inadequate, look at their source code.
|