mirror of
https://github.com/wassname/ml-debug.git
synced 2026-09-26 14:00:26 +08:00
wassname
60ed9df651
The notes file listed 11 links; 9 are already in the full page cache and the 80k Hours one has its own cache, so it only carried Rocktaschel, now in the research_taste reading list. The Nanda shared draft was a 902-word excerpt of a local download; wassname supplied the Google Doc, whose text export gives the full 11318 words.
46 lines
3.4 KiB
Markdown
46 lines
3.4 KiB
Markdown
# Spinning Up as a Deep RL Researcher - source graph and research-taste excerpts
|
|
|
|
Primary source: https://spinningup.openai.com/en/latest/spinningup/spinningup.html
|
|
Author: Joshua Achiam, OpenAI
|
|
Date: October 13th, 2018
|
|
Related local cache: docs/evidence/spinningup_researcher.md
|
|
Fetch-status: index only. The full page text now lives in docs/evidence/spinningup_researcher.md (fetched 2026-08-15); this file keeps the source graph so the two caches do not hold the same 3.3k words twice (CLAUDE agent)
|
|
Use: RL research-process evidence, especially for source graph, fair comparisons, seeds, preregistration, and ablations.
|
|
|
|
## Why this matters for agents
|
|
|
|
Spinning Up is not just an RL textbook page. Its researcher page is a compact research apprenticeship guide. It points to the same battle-tested debugging and reproducibility references already cached in this repo, then adds project selection and rigorous comparison advice.
|
|
|
|
## Quotes
|
|
|
|
The quotes that used to sit here (simplicity is critical, iterate fast in simple
|
|
environments, avoid reinventing the wheel, handicap the baseline, seeds,
|
|
preregistration, ablations) are all in the full page text at
|
|
[spinningup_researcher.md](spinningup_researcher.md). Read that file for the
|
|
wording; this one only carries the source graph. (CLAUDE agent, 2026-08-15)
|
|
|
|
## Source graph
|
|
|
|
Spinning Up intro references, with local status:
|
|
- Alex Irpan, Deep Reinforcement Learning Doesn't Work Yet: https://www.alexirpan.com/2018/02/14/rl-hard.html. Local cache: docs/evidence/alexirpan_rl_hard.md.
|
|
- Islam et al., Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control: https://arxiv.org/pdf/1708.04133. Not separately cached; discussed/cited inside Henderson local cache.
|
|
- Henderson et al., Deep Reinforcement Learning that Matters: https://arxiv.org/pdf/1709.06560. Local cache: docs/evidence/henderson_2018_deep_rl_matters.md.
|
|
- Matthew Rahtz, Lessons Learned Reproducing a Deep RL Paper: http://amid.fish/reproducing-deep-rl. Local cache: docs/evidence/amid_fish_reproducing_deep_rl.md.
|
|
- David Silver UCL RL course: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html. Not cached.
|
|
- Berkeley Deep RL course: http://rll.berkeley.edu/deeprlcourse/. Not cached.
|
|
- Deep RL Bootcamp lectures: https://sites.google.com/view/deep-rl-bootcamp/lectures. Reddit index cache: docs/evidence/reddit_deeprl_bootcamp_2017_75m5vd.md.
|
|
- John Schulman, Nuts and Bolts of Deep RL: http://joschu.net/docs/nuts-and-bolts.pdf. Local cache: docs/evidence/joschu_nuts_and_bolts.md.
|
|
- Tim Rocktaschel et al., Advice for Short-term Machine Learning Research Projects: https://rockt.github.io/2018/08/29/msc-advice.html. Not cached.
|
|
- Catherine Olsson / 80,000 Hours, ML Engineering for AI Safety & Robustness: https://80000hours.org/articles/ml-engineering-career-transition-guide/. Not cached.
|
|
|
|
## Likely follow-up cache candidates
|
|
|
|
Priority 1:
|
|
- Chris Olah, research taste: short and directly named by Nanda.
|
|
- Jacob Steinhardt, Research as a Stochastic Decision Process: directly named by Nanda for prioritization.
|
|
- Tim Rocktaschel et al., short-term ML research projects: directly named by Spinning Up for research growth.
|
|
|
|
Priority 2:
|
|
- David Silver/UCL, Berkeley Deep RL, Deep RL Bootcamp: curriculum material, less directly research-taste except via RL mastery.
|
|
- Catherine Olsson/80k: career/field-entry framing; useful if the skill expands beyond project-level research taste.
|