From 54e7708f157ac209aa9712918ebdeae68227dd53 Mon Sep 17 00:00:00 2001
From: wassname <1103714+wassname@users.noreply.github.com>
Date: Wed, 26 Aug 2026 14:23:48 +0800
Subject: [PATCH] exercise 12: swap the tank legend for Zech et al. and the
 fastbook grant leak

gwern's own page concludes the tank story did not happen, so citing it
undercut the exercise. Zech is peer-reviewed with the in-site against
out-of-site AUC pair; the fastbook case covers the tabular version.
---
 SKILL.md | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/SKILL.md b/SKILL.md
index 567398b..081e270 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -247,7 +247,9 @@ For each spike or collapse, show the log rows before it. Say which column moved
 
 ## 12. "The NN had learned something useless like time of day" (small)
 
-> Researchers training a neural network to detect tanks in photographs, succeeding, only to realize the photographs had been collected under specific conditions for tanks/non-tanks and the NN had learned something useless like time of day. -- gwern, who traced it back to 1992 and calls it an urban legend
+> The CNN has learned to detect a metal token that radiology technicians place on the patient in the corner of the image field of view at the time they capture the image. -- Zech et al., whose pneumonia model scored AUC 0.931 in its own hospitals and 0.815 in someone else's
+
+> The model was able to correctly predict who would receive grants over 95% of the time. Apparently meaningless identifier columns were the most important predictors. [...] It turned out that in practice, the university only filled out much of this information *after* a grant application was accepted. -- Howard and Gugger
 
 For the headline metric, name one useless thing the model can learn and still score well, for
 example a condition of data collection or the class prior. Show the control arm or the row that