mirror of
https://github.com/wassname/ml-debug.git
synced 2026-06-27 19:00:00 +08:00
wassname
8509ec3c30
- Promote the general (non-RL-specific) Spinning Up lessons up to the main folklore: "broken code fails silently", "you can't tell it's broken if you can't see that it's breaking", and test on more than one setup. - Add gwern's "Unseeing" to the data theme: you can't read what you actually wrote, hence fresh eyes / a fresh-eyes subagent. - New "Research taste (adjacent to debugging)" section with verbatim quotes, each cached: Neel Nanda (your research is false by default; excitement is evidence of bullshit; read your data), Ulisse Mini (understand the system to shrink the search space), John Wentworth (gears-level models are capital investments vs cheap black boxes). All quotes verbatim from cached sources; 25/25 footnotes resolve. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
818 B
818 B
How to get good at programming — Ulisse Mini
Source: https://www.lesswrong.com/posts/LTypqBMTSmRrrhb2v/how-to-get-good-at-programming . Verbatim excerpts cached for the skill.
When good programmers debug hard problems fast, it's usually because they understand the system well enough to track the important internal state in their head, letting them drastically reduce the solution space they're searching over.
you must notice when you're going into brute-force search mode, and then take action by investing time in understanding the underlying system, until both the problem and solution make sense.
It is higher value to white-box leaky abstractions. Autograd for ML is a great example of a leaky abstraction, if you mix up
permuteandviewyour gradients can be subtly wrong.