Commit Graph
174 Commits
Author SHA1 Message Date
wassnameandPI[openai-codex] 1df150618a refresh dev4 skill draft
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 08:33:45 +08:00
wassnameandPI[openai-codex] 006ee0ae4d try compact research-loop skill
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 08:27:26 +08:00
wassnameandPI[openai-codex] 4e1e77fa24 add Agans nine rules: full verbatim quotes + evidence notes
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 07:21:28 +08:00
wassnameandPI[openai-codex] 0c102d5177 add Tobin debugging sequence
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 06:59:55 +08:00
wassnameandPI[openai-codex] b7f46074fa make design review tool-independent
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 06:47:31 +08:00
wassnameandPI[openai-codex] 2c62449d9b fix skill install link and enforce audits
Co-Authored-By: PI[openai-codex] <288921227+claudypoo@users.noreply.github.com>
2026-09-02 06:32:09 +08:00
wassnameandClaudypoo 6106575e9c replace grader-folk words with the precise term
knob -> hyperparameter, gate -> threshold, rung -> defined level, clean log ->
error-free log, ship the rubric -> put the rubric. Quoted source text (Irpan's
canary, Domingos' knobs) and wassname's own lines are untouched.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-09-01 05:46:15 +08:00
wassnameandClaudypoo e4ae445108 form is an anti-skim gate: task/eval framing, scoring rows, routing is mandatory
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-31 17:30:35 +08:00
wassname 8c62f961a8 restore wassname's original form framing: Q: prefixes, Is it: nesting, TODOs, evaluated-on line 2026-08-31 17:29:48 +08:00
wassnameandClaudypoo 79f46be733 add a fill-in form at the head, so an agent reading only the top does one core exercise
Long form stays in ex #1/#3/#7/#15; the form is self-contained.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-31 17:27:30 +08:00
wassname 37bb6fcf90 two more quotes at the top, and why Rahtz's fast-feedback argument does not transfer to an agent 2026-08-27 19:43:20 +08:00
wassname cb03fb18fd the v9 row: be-diligent-first scores +0.56, above bare, where the old text was below it 2026-08-27 18:00:38 +08:00
wassname 7451008c1b README table: point the untested row at the version that will be tested 2026-08-27 09:59:19 +08:00
wassnameandClaudypoo 3a58c54170 follow the skill spec: references/ not refs/, and namespaced subskill names
- refs/ -> references/, the folder name the Agent Skills spec uses and the one
  Hermes skips when it walks for nested skills.
- rl and pinn declared name: rl and name: pinn, which are global names in a flat
  skill namespace. Now ml-debug-rl and ml-debug-pinn. They also called themselves
  sub-skills of 'ml-debugging', which is not this skill's name.
- Drop the dead link to SKILL_old.md. It moved into gitignored slop/, so the link
  was broken for anyone who cloned.
- Route references/llm_judge_litreview.md, the one reference SKILL.md never named.
- Description leads with the trigger situations. Hermes truncates it to 57 chars.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-27 09:59:01 +08:00
wassname 739f846297 README: which part of the document does the work, and where each version lives 2026-08-27 09:56:43 +08:00
wassname 26ed3b80b7 misc 2026-08-27 09:48:46 +08:00
wassname 742dbe4d41 keep the bench numbers out of the skill, point at the README table 2026-08-27 09:48:21 +08:00
wassname 6e149bde5e put be-diligent at the top, it is the part with measured uplift 2026-08-27 09:45:19 +08:00
wassname e296293960 name the exercises, and make the selector a nested if/then list
Headings were the practitioner quote; the quote still opens each exercise, the
heading is now what the exercise makes you do (ex #3 read your data). Every
cross-reference carries the name, so 'Ex #7' never appears bare.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-27 09:19:39 +08:00
wassname bc136d224e wassname's rule: metrics and demos inline in the train script, no side-car probes
Also turns the exercise selector into an explicit if/then table. 7 and 8 were
bundled under 'about to report a result'; they now have their own conditions.
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-27 08:35:11 +08:00
wassname efcac5ca5f plain wording: do the exercises and show the result 2026-08-26 16:14:24 +08:00
wassname b8689ad23b fail fast is advice for a human who over-commits, agents quit early instead 2026-08-26 16:09:00 +08:00
wassname 392da00cc4 Common mistakes: promote the crash-loudly rule from PLAYBOOK, plus Nanda
Two different rules share the name fail fast. Nanda's is killing a doomed
direction early; this one is crashing on the error instead of carrying on.
Kept apart on purpose.
2026-08-26 15:58:58 +08:00
wassname 8429a08903 sign off: name who said the quote 2026-08-26 15:44:47 +08:00
wassname fe2d13bf4d SKILL.md: extend Karpathy quote to what to do when out of ideas
The order alone is unfollowable for an agent that has genuinely run dry.
Left out the 5-minute-experiment arithmetic, which gives the wrong number
for hour-long architecture runs.
2026-08-26 15:42:50 +08:00
wassname 986c017fce SKILL.md: Karpathy's NEVER STOP next to wassname's never-give-up line 2026-08-26 15:34:40 +08:00
wassname bfb97f061c SKILL.md: wassname's note on field-standard language fills the LLM slot
His text, spelling fixed and voice kept, plus the three quotes from the cache
that back it: Bekman flagging his own overloaded heading, the tuning playbook
on two things sharing the name learning_rate, and Lones on which AUC.
2026-08-26 15:23:08 +08:00
wassname 5ab2418c3c exercise 12: heading follows the new quote 2026-08-26 14:24:03 +08:00
wassname 54e7708f15 exercise 12: swap the tank legend for Zech et al. and the fastbook grant leak
gwern's own page concludes the tank story did not happen, so citing it
undercut the exercise. Zech is peer-reviewed with the in-site against
out-of-site AUC pair; the fastbook case covers the tabular version.
2026-08-26 14:23:48 +08:00
wassnameandClaudypoo d7536fe549 SKILL.md: annoy-less comment review on the AI-written prose
Comment review mode only, no prose changed. Flags negative framing,
aphoristic closers, and three places where the rewrite made wassname's
hedged claims stronger than his original message.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-26 14:23:18 +08:00
wassnameandClaudypoo aa0f45de80 SKILL.md: add empty slot for wassname's note on LLM agents
Textbook order: collected advice, then his comment on how it applies to
LLMs, then the exercises. Content left for him to write.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-26 14:21:58 +08:00
wassnameandClaudypoo d4cad35f42 SKILL.md: mark each exercise small or large, route by size
Small is under a paragraph; large means work like comparing against a
reference repo. Do all applicable small ones, pick one large one.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-26 14:21:51 +08:00
wassname cafc1c89be SKILL.md: quote Sanh on the threshold mistake, the one bullet left bare 2026-08-26 14:16:25 +08:00
wassname d769cacfd4 README: add the eight common mistakes with 46 mined quotes
Quotes come from the docs/evidence cache and were previously unused. Reuses the
existing footnote style, 14 new keys. Carries the source doc's coverage warning:
mode 6 has only three quotes and mode 7 has none that name similarity probes.
2026-08-26 13:47:00 +08:00
wassname 3cc4ca3eb7 SKILL.md: add wassname's eight common mistakes, with the Nanda, Sanh and Achiam quotes
Source is his own message of 2026-08-25, spelling fixed and slightly more polite as he
asked, with each mistake pointing at the exercise that answers it. Also adds his rule
that a job is never abandoned without doing the exercises, one at a time.
2026-08-26 13:44:52 +08:00
wassnameandClaudypoo 6351958834 add exercises 14 and 15: one failed attempt is not a negative, and get the scale before the gate
Two gaps the existing 13 did not cover, found by mining the evidence cache against
wassname's list of common AI-agent failures. Quotes are verbatim from
docs/evidence/ (Steinhardt, Rahtz, Nanda, Goodfellow-Bengio-Courville).

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-26 13:32:48 +08:00
wassname 3d84683036 Revert "SKILL.md: adopt v6 (terra's seven exercises plus wassname's common mistakes with sources)"
This reverts commit 765a061fb7.
2026-08-26 08:24:11 +08:00
wassnameandClaudypoo 765a061fb7 SKILL.md: adopt v6 (terra's seven exercises plus wassname's common mistakes with sources)
Measured on wassname-ml-bench v97, 12 items, loaded-skill header:
  grok-4.6   bare +0.592, v5 +0.809, v6 +0.785
  deepseek   bare +0.706, v5 +0.705, v6 +0.737
v5 and v6 are indistinguishable there; v6 carries wassname's failure modes.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-26 08:08:16 +08:00
wassnameandClaudypoo 26eb2cce6a candidate quotes and a common-mistakes draft, not yet in the README
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-25 11:23:30 +08:00
wassname 192311c425 skill: sign off with a quote in a hand-drawn ascii animal 2026-08-24 22:16:42 +08:00
wassname 2871d89512 skill: the shorter runbook, 174 lines from 255 2026-08-24 21:39:04 +08:00
wassnameandClaudypoo dc369f5fac skill: fold in the design-doc gaps -- outcome signatures, persisted predictions, executed config, sample selection, guide caps
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-20 06:53:38 +08:00
wassnameandClaudypoo 2e1eefbba6 skill: raw artifacts in P1, seed-noise and baseline before A-beats-B, fresh auditor, follow the job
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-20 06:46:23 +08:00
wassnameandClaudypoo c94a450518 skill: permit 'unknown', require attribution when several things change at once
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-20 06:45:09 +08:00
wassnameandClaudypoo ff797a95e1 skill: keep failure interpretations, ban failure causes; history goes to the journal
Interpretations must read as evidence for or against a claim, never as a cause,
because a cause list gets picked from, called certain, and used to stop.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-20 06:41:02 +08:00
wassnameandClaudypoo e2bd28dbc2 skill: read your data and assume you have a bug were only in the frontmatter
Both were promised by the description and absent from the procedure. P2 now
prints the formatted examples; P3 opens with the bug assumption.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-20 06:39:36 +08:00
wassnameandClaudypoo f830c0cb23 skill: put literal trigger phrases in the description so it self-invokes
A sonnet subagent had ml-debug listed, with 'after a run finishes or crashes' in
the description, and did not invoke it for 'read the last pueue log'.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-19 20:55:43 +08:00
wassnameandClaudypoo a7ff779e88 skill: P1 steps 1-4 are read-only, so a 'just read the log' task still fills the table
Sonnet routed to P1 correctly then declined it: 'the task only asked to read, not
to audit or act'. Reading a log is exactly when the measurement table gets filled.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-19 20:52:56 +08:00
wassnameandClaudypoo f3f1a38485 skill: rewrite as a runbook -- P1-P5, numbered imperative steps, stop gates
Sonnet read the block version and summarised it instead of running it.

Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-19 20:51:10 +08:00
wassnameandClaudypoo c858469f59 readme: match the skill's wording, instructions not rituals
Co-Authored-By: Claudypoo <288921227+claudypoo@users.noreply.github.com>
2026-08-19 20:46:48 +08:00