wassname/ml_debug
1
0
Fork 0
mirror of https://github.com/wassname/ml_debug.git synced 2026-09-09 11:27:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

add exercises 14 and 15: one failed attempt is not a negative, and get the scale before the gate

Browse Source 
Two gaps the existing 13 did not cover, found by mining the evidence cache against
wassname's list of common AI-agent failures. Quotes are verbatim from
docs/evidence/ (Steinhardt, Rahtz, Nanda, Goodfellow-Bengio-Courville).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
wassname
2026-08-26 13:32:48 +08:00
co-authored by Claudypoo
parent 3d84683036
commit 6351958834
1 changed files with 32 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SKILL.md
+32 -1
View File
@@ -12,7 +12,8 @@ do exercises 1, 3 and 7. Then select by situation:
- something weird in the log (a spike, a flat line, an impossible value): 10, 11
- about to queue a run: 5, 6
- about to change the design, or a run you cannot explain: 13
- about to report a result, or to call it negative: 7, 8, 12
- about to report a result, or to call it negative: 7, 8, 12, 14
- about to set a pass gate or quote a threshold: 15
- two cycles with no progress: 9
Each exercise says what to show. Show it in full: the table, the quoted log line, the quoted
@@ -153,6 +154,36 @@ forward pass and the backward pass. Show all three. Send them to `/external-revi
scientist mode and show the verdict. The reviewer sees only the description, so make the
description complete.
## 14. "An implementation comprising 0.1% of the possible implementations of X"
> Trying an experiment and seeing it fail gives little information by itself. When an experiment fails, it is tempting to conclude "I tried X and it didn't work". However, if X is a high-level conceptual approach, then a more correct conclusion is "I tried an implementation comprising 0.1% of the possible implementations of X, and observed that that particular implementation did not work". -- Steinhardt
> It ended up taking me 6 weeks to reproduce results, thanks to several software bugs. The question is, why did it take so long to find these bugs? -- Rahtz
Before you call an idea dead, show the implementation you actually ran and one other
implementation of the same idea that you did not run. Say what would have to be true for the
idea to be alive and your run to still fail. Then do exercise 7 on your own code before you
write the negative up.
| the idea | what I ran (file:line) | one other way to run it | what a bug here would look like |
|---|---|---|---|
One attempt is untested, not negative. Say which of the two this is.
## 15. "By default, all numbers are meaningless because we lack any scale"
> A valuable intuition to have in mind is that, by default, all numbers are meaningless because we lack any scale to compare them. E.g. if a probe gets 95% classification accuracy on some task, is this good? Is this bad? Hard to say without knowing more! Baselines are one way to get context to compare against. -- Nanda
> In most cases, we do not know a priori what the intended behavior of the algorithm is. [...] If we train a neural network on a new classification task and it achieves 5 percent test error, we have no straightforward way of knowing if this is the expected behavior or suboptimal behavior. -- Goodfellow, Bengio and Courville
Before you set a pass gate or quote a threshold, get the scale first. Run the metric on a null
arm, a shuffled or permuted control, and the existing baseline, then set the bar against those.
| metric | null arm | shuffled control | current baseline | ceiling the data allows | proposed gate |
|---|---|---|---|---|---|
A gate chosen before this table is a number you made up. Say so if you have to use one anyway.
## Reference
Sources and more quotes: [README.md](README.md). Longer material, open the one you need: