Commit Graph
5 Commits
Author SHA1 Message Date
wassname ceba01782b replace four invented nanochat quotes with Karpathy's own words
All four came from the AI-generated deepwiki page, not the experiment log,
and one inverted the finding: the log says clipping was removed because
'Grad norm never exceeds 1.0 naturally', and the real distributed gotcha was
clipping local norms before sync. Footnote now points at dev/LOG.md on master
and at the renamed cache.
2026-08-15 06:22:53 +08:00
wassname a6c8ba77d2 cite arxiv pdf, never abs
An /abs/ link is a stub: it costs a second lookup before anyone can check the
quote, and it is what let abstract-only caches look sourced. 77 links across
the skill files and 4 cache headers. Verbatim source bodies untouched, their
reference lists are the authors' text.
2026-08-15 06:14:10 +08:00
wassnameandClaudypoo 4e45bd4140 DPO can barely move: tinker's own reference run reports 0.52 pair accuracy
Contrast with the cookbook's RLHF pipeline (46% -> 94% win rate in 100 steps),
so a decreasing DPO loss is weak evidence of behaviour change.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-08 10:04:32 +08:00
wassname 160bd040cc docs: add sourced transformer report folklore 2026-06-12 07:02:58 +08:00
wassname 8b9a1d62ed docs: resolve ml-debug TODO references 2026-06-12 06:52:38 +08:00