Commit Graph

Select branches

Hide Pull Requests

benchmark/skill-ab

main

Mono Color

• 996942c4de Make ML debugging expose evidence and uncertainty main wassname 2026-08-09 11:11:02 +08:00
• 4e45bd4140 DPO can barely move: tinker's own reference run reports 0.52 pair accuracy wassnameandClaudypoo 2026-08-08 10:04:32 +08:00
• b5a70354a4 Judgemark: add the cheap-tier rows and date the snapshot wassnameandClaudypoo 2026-08-06 08:24:01 +08:00
• 602f6193ed assume every negative result is a bug until the logs rule it out; be patient wassnameandClaudypoo 2026-07-28 17:14:05 +08:00
• c7f22b8478 Add deployment-faithful time-series evaluation guide wassname 2026-07-26 10:31:15 +08:00
• a9b154fc6d add FPP3 (Hyndman & Athanasopoulos) scraped to markdown, 13 chapters + appendices wassname 2026-07-26 10:21:48 +08:00
• 4fcea49ebb Show explicit pandas failure-score aggregation wassname 2026-07-26 09:48:31 +08:00
• 0cc01db100 Clarify NaN selection bias in judge comparisons wassname 2026-07-26 09:46:42 +08:00
• 5dd18385bd Fail judge aggregation on missing paired scores wassname 2026-07-26 09:42:18 +08:00
• 2c009b85b6 llm_judges: add Petri judge-harness patterns (rubric in schema, tier anchors, resolvable citations, harness dimensions, scanner-as-scorer, fail-open anti-pattern) wassnameandClaudypoo 2026-07-25 10:15:36 +08:00
• 38ddb544d9 Make 'Numbers worth knowing' quote-anchored with source-credence + calibrated language wassnameandClaudypoo 2026-07-23 12:02:51 +08:00
• 3f35addb7f Expand 'Numbers worth knowing' bullets with point + context for readability wassnameandClaudypoo 2026-07-23 11:57:53 +08:00
• 23d3331ee2 Fix two varglite nits: complete Lost-in-Middle quote, add context to config block wassnameandClaudypoo 2026-07-23 11:48:50 +08:00
• f10151e661 Add varglite 2026 judge lit-review + operational-numbers digest wassnameandClaudypoo 2026-07-23 11:46:12 +08:00
• 3b7e82d402 Add 2026 lit-search batch to judge-bias evidence (position/self/reasoning/context-rot + benchmark index) wassnameandClaudypoo 2026-07-23 09:19:06 +08:00
• 2692ea8a4e Add reasoning-token budget guidance and LLM-as-judge survey wassnameandClaudypoo 2026-07-23 08:54:26 +08:00
• 01db376bca Add real SpeechMap per-lab refusal index to judge-selection wassnameandClaudypoo 2026-07-22 16:16:42 +08:00
• 54b8a2bb79 Add real Judgemark v4 frontier and evaluator-blind-spot papers wassnameandClaudypoo 2026-07-22 16:12:46 +08:00
• 0105153305 Add judge-validity checklist and model-selection to llm_judges ref wassnameandClaudypoo 2026-07-22 16:09:18 +08:00
• 9d00a069c6 Require reading both sides of LLM evaluations wassname 2026-07-18 14:41:59 +08:00
• cdab6cbbb9 Record ML debugging benchmark results benchmark/skill-ab wassname 2026-07-13 06:11:08 +08:00
• a0fcfa291e Add blinded ML debugging benchmark harness wassname 2026-07-12 22:18:26 +08:00
• 9774c4bb1d Record skill audit evidence wassname 2026-07-12 22:01:36 +08:00
• 0be4323312 Add fail-fast skill audit wassname 2026-07-12 22:00:30 +08:00
• fa534cf44e Correct overconfident debugging advice wassname 2026-07-12 22:00:25 +08:00
• e92ec01efe Enhance ML debugging guidance for LLM agents wassname (Michael J Clark)andGitHub 2026-06-26 09:52:43 +08:00
• 5fca5ad2b2 Refresh Schulman cache anchors after transcript rewrite wassnameandClaudypoo 2026-06-25 10:28:23 +08:00
• f8f512f603 Cite Irpan in research taste (signs-of-life, seed canary) wassnameandClaudypoo 2026-06-25 10:26:10 +08:00
• 3fe6cb9ad9 Replace OCR-garbled Schulman cache with clean slide transcript wassnameandClaudypoo 2026-06-25 10:26:10 +08:00
• 67d4dc90bb Document quote-first evidence style wassname 2026-06-25 09:56:26 +08:00
• 20f03f20b8 Expand research taste appendix with expert quotes wassname 2026-06-25 09:47:39 +08:00
• 8fc2c0bbd0 Add research taste evidence appendix wassname 2026-06-25 09:33:34 +08:00
• 3f3f95a3b4 Update SKILL.md with debugging strategies and folklore wassname (Michael J Clark)andGitHub 2026-06-14 08:37:57 +08:00
• b8c3ffcf11 gpt5.5/fable wassname 2026-06-12 09:30:25 +08:00
• 160bd040cc docs: add sourced transformer report folklore wassname 2026-06-12 07:02:58 +08:00
• 3e28a950e9 docs: clarify competing-worlds debugging loop wassname 2026-06-12 06:53:36 +08:00
• 8b9a1d62ed docs: resolve ml-debug TODO references wassname 2026-06-12 06:52:38 +08:00
• 966f948d36 docs: refine numerical and scheduler debugging guidance wassname 2026-06-12 06:35:23 +08:00
• e58eda360b Update SKILL.md with TODOs for future content wassname (Michael J Clark)andGitHub 2026-06-11 21:35:49 +08:00
• 1ad74e14c6 Update loss_surface.md wassname (Michael J Clark)andGitHub 2026-06-11 21:21:01 +08:00
• 6e9a3ca633 Revise introduction in diagnostics.md wassname (Michael J Clark)andGitHub 2026-06-11 21:18:07 +08:00
• 30ac76053e chore: drop links to deleted/tombstone gists (repo is canonical now) wassnameandClaudypoo 2026-06-11 16:55:04 +08:00
• 0837f27f08 fix: companion gist link pointed at the wrong gist wassnameandClaudypoo 2026-06-11 16:41:50 +08:00
• 8cd3c61050 folklore: tuning playbook, Domingos, Bekman loss spikes, Ng error analysis; LLM-judge bias appendix wassnameandClaudypoo 2026-06-11 15:30:41 +08:00
• 2a2f5045bb folklore: add Karpathy common-mistakes tweet and Sculley CACE principle wassnameandClaudypoo 2026-06-11 14:43:47 +08:00
• fb753d093e restructure: quotes-first SKILL.md, synthesized playbook split out wassnameandClaudypoo 2026-06-11 14:33:32 +08:00
• 8ee980d62f diagnostics: add NaN-poisoning leakage tracer + Karpathy backprop-to-input check; README citation wassnameandClaudypoo 2026-06-11 10:18:51 +08:00
• 53fb1c4bda Update SKILL.md wassname (Michael J Clark)andGitHub 2026-06-03 08:19:07 +08:00
• 8e1f9dec6d Update README.md wassname (Michael J Clark)andGitHub 2026-06-03 08:18:11 +08:00
• d1b1608a49 Update README.md wassname (Michael J Clark)andGitHub 2026-06-03 08:17:04 +08:00
• 8509ec3c30 folklore: promote Spinning Up to main; add a Research-taste section wassnameandClaudypoo 2026-06-02 21:08:49 +08:00
• a602ea5a0e rl: quote Spinning Up (Achiam) on silent failure and bug-first debugging wassnameandClaudypoo 2026-06-02 21:04:55 +08:00
• ee4e9a5caa folklore: add koaning, gwern, kidger, nanochat, cleanrl; trim lucidrains wassnameandClaudypoo 2026-06-02 20:59:36 +08:00
• 9911ac83c5 folklore: add lucidrains transformer-stability item (QK-norm, post-emb LN) wassnameandClaudypoo 2026-06-02 20:49:15 +08:00
• 38ec634ff3 restructure: folklore-first, quote-verified, with wassname intro wassnameandClaudypoo 2026-06-02 20:46:25 +08:00
• cf9df71f6a add SKILL2.md: condensed anchor proposal (74 vs 703 lines) wassnameandClaudypoo 2026-06-02 13:26:59 +08:00
• ab827116b5 remove AI flourishes and rhetorical "X, not Y" framing wassnameandClaudypoo 2026-06-02 12:56:35 +08:00
• 7410a7ccf3 restore -- attribution form for blockquote citations wassnameandClaudypoo 2026-06-02 12:52:43 +08:00
• b6fad64930 loop pseudocode: pseudopy style (← assignment, ── divider, t̂) wassnameandClaudypoo 2026-06-02 12:51:15 +08:00
• 90b11214f8 de-AI pass: drop em-dashes, flourishes; resolve in-file TODOs wassnameandClaudypoo 2026-06-02 12:49:28 +08:00
• 220bd8dc7f fix typos: separate/reproduced/auditable, drop stray article wassnameandClaudypoo 2026-06-02 12:07:58 +08:00
• 715164416b loop: add likelihood-ratio test selection, path bisection, falsifiers, pseudocode wassnameandClaudypoo 2026-06-02 12:06:30 +08:00
• d5c7dec5a6 Update SKILL.md wassname (Michael J Clark)andGitHub 2026-06-01 13:36:35 +08:00
• 779beee03e refactor(ml_debug): tidy ordering/emphasis on the new top sections wassnameandClaudypoo 2026-06-01 10:15:41 +08:00
• bb1a6bc61c feat(ml_debug): lead with judgment gates over the symptom-lookup encyclopedia wassnameandClaudypoo 2026-06-01 10:11:36 +08:00
• fde5ac62fd name wassname 2026-04-09 05:09:25 +08:00
• b159b0fba8 docs(ml_debug): annotate EMNLP 2018 NLP code tutorial; note sparse Adam embedding bug wassname 2026-03-10 05:48:36 +08:00
• 0fa4009fd5 docs(ml_debug): update Grus annotation after reading full slides; note EMNLP 2018 lead wassname 2026-03-10 05:45:56 +08:00
• 52ff6c17cd docs(ml_debug): annotate Joel Grus slides -- SE/reproducibility talk, not debugging wassname 2026-03-10 05:45:16 +08:00
• 3dffe890b1 docs(ml_debug): annotate sanh outbound links with content summaries wassname 2026-03-10 05:40:31 +08:00
• c9c53f8e7f feat(ml_debug): expand nanochat evidence, add lec4 diagnostics file wassname 2026-03-10 05:38:33 +08:00
• ced4edc200 feat(ml_debug): add Karpathy recipe + nanochat evidence, update-ratio diagnostic wassname 2026-03-10 05:32:37 +08:00
• bbe3fe0985 feat(ml_debug): add JAX grep patterns and diagnostic equivalents wassname 2026-03-06 14:10:39 +08:00
• 7ac7aacac7 fix(ml_debug): address review feedback wassname 2026-03-06 13:59:48 +08:00
• 70c28f06ac refactor(ml_debug): extract grep patterns and diagnostics to refs/ wassname 2026-03-06 13:54:37 +08:00
• 48d4c1044a refactor(pinn): extract heat exchanger specifics to refs/ wassname 2026-03-06 13:39:53 +08:00
• 7f34f26a5c refactor(ml_debug): extract RL debugging into rl/ sub-skill wassname 2026-03-06 13:36:29 +08:00
• 698b77f2d3 chore: fix .gitignore (dlbooks path, *_log.md pattern) wassname 2026-03-06 12:22:22 +08:00
• 9e30cf7039 chore: remove duplicate subtitle file and log (now gitignored) wassname 2026-03-06 12:21:48 +08:00
• fa41fecef2 Delete docs/dlbooks wassname (Michael J Clark)andGitHub 2026-03-06 12:19:12 +08:00
• 7a9c667aa7 chore: add wassname attribution to description, gitignore dlbooks wassname 2026-03-06 12:17:50 +08:00
• 463c8fdbbc fix: apply Gemini review fixes (device kwarg, gradcheck requires_grad, torch prefix) wassname 2026-03-06 12:15:37 +08:00
• 2db012dd2c docs(pinn): add Wang 2021 and Rathore 2024 evidence files wassname 2026-03-06 12:12:51 +08:00
• a90624b36d feat(pinn): add pinn/ sub-skill with SKILL.md and evidence wassname 2026-03-06 11:48:41 +08:00
• 51c9a2df44 docs: add README with author credit and usage wassname 2026-03-06 10:16:24 +08:00
• 95fee7b5cb chore: include Goodfellow chapters (author encourages sharing) wassname 2026-03-06 10:16:00 +08:00
• 4393cceefd initial: ML debugging folklore skill wassname 2026-03-06 10:11:30 +08:00