mirror of
https://github.com/wassname/ml_debug.git
synced 2026-09-09 11:27:07 +08:00
follow the skill spec: references/ not refs/, and namespaced subskill names
- refs/ -> references/, the folder name the Agent Skills spec uses and the one Hermes skips when it walks for nested skills. - rl and pinn declared name: rl and name: pinn, which are global names in a flat skill namespace. Now ml-debug-rl and ml-debug-pinn. They also called themselves sub-skills of 'ml-debugging', which is not this skill's name. - Drop the dead link to SKILL_old.md. It moved into gitignored slop/, so the link was broken for anyone who cloned. - Route references/llm_judge_litreview.md, the one reference SKILL.md never named. - Description leads with the trigger situations. Hermes truncates it to 57 chars. Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# Unused quotes from the ml-debug evidence cache
|
||||
|
||||
Mined from `/home/wassname/.agents/skills/ml-debug/docs/evidence/` (about 40 cached sources) and
|
||||
`/home/wassname/.agents/skills/ml-debug/refs/`. Every quote here was checked against
|
||||
`/home/wassname/.agents/skills/ml-debug/references/`. Every quote here was checked against
|
||||
`/home/wassname/.agents/skills/ml-debug/README.md` and is not used there. Line numbers were
|
||||
verified by grep on a distinctive substring; long source lines are single wrapped paragraphs, so
|
||||
one line number can hold a long quote.
|
||||
@@ -82,7 +82,7 @@ Why it lands: seed noise alone can clear a significance bar. So one A-versus-B g
|
||||
Why it lands: turns "am I overconfident" into one answerable question with a calibration target, and points the next action at the least reliable step rather than the most interesting one.
|
||||
|
||||
## My Model of the Research Process (shared draft), as quoted in the skill's own topic note -- Neel Nanda
|
||||
- file: /home/wassname/.agents/skills/ml-debug/refs/research_taste.md:134
|
||||
- file: /home/wassname/.agents/skills/ml-debug/references/research_taste.md:134
|
||||
- failure modes: 1, 3
|
||||
- epistemic context: quoted from an unpublished Google Doc draft, so weaker provenance than the published posts by the same author.
|
||||
|
||||
@@ -224,7 +224,7 @@ Why it lands: two modes at once. Hypotheses 2 and 3 can be hypothesis 1 wearing
|
||||
Why it lands: a symptom-to-cause table where every symptom has two or three candidates and only one of them is a learning rate. It is a ready-made hypothesis-2-and-3 generator for the moment the agent reaches for the knob.
|
||||
|
||||
## My Model of the Research Process (shared draft), as quoted in the skill's own topic note -- Neel Nanda
|
||||
- file: /home/wassname/.agents/skills/ml-debug/refs/research_taste.md:120
|
||||
- file: /home/wassname/.agents/skills/ml-debug/references/research_taste.md:120
|
||||
- failure modes: 3
|
||||
- epistemic context: unpublished draft quoted in a local topic note; weaker provenance than the published posts.
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ summarizer produced from a web page, and nobody has read the paper. On
|
||||
2026-08-15 I re-pulled the five [ID] entries the litreview depends on and two
|
||||
of the five carried a wrong number, so treat the remaining 11 as roughly 2-in-5
|
||||
wrong until each is checked against raw text. Do not promote an [ID] number
|
||||
into SKILL.md or refs/ without re-pulling the paper first.
|
||||
into SKILL.md or references/ without re-pulling the paper first.
|
||||
|
||||
## "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" — Zheng et al. (LMSYS), NeurIPS 2023 — https://arxiv.org/pdf/2306.05685
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ Source: https://arxiv.org/pdf/2411.00640 (Evan Miller, Anthropic, Nov 2024) + ht
|
||||
Title: Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
|
||||
Fetched-via: r.jina.ai on the arXiv PDF and the Anthropic post, 2026-08-16
|
||||
Fetch-status: verbatim from full PDF text (math notation mangled by the PDF-to-markdown pass; prose is clean)
|
||||
Used-by: refs/llm_judges.md (repeat draws, temperature, paired differences)
|
||||
Used-by: references/llm_judges.md (repeat draws, temperature, paired differences)
|
||||
|
||||
# Adding Error Bars to Evals (excerpts)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user