wassname
60ed9df651
The notes file listed 11 links; 9 are already in the full page cache and the 80k Hours one has its own cache, so it only carried Rocktaschel, now in the research_taste reading list. The Nanda shared draft was a 902-word excerpt of a local download; wassname supplied the Google Doc, whose text export gives the full 11318 words.
3.4 KiB
Spinning Up as a Deep RL Researcher - source graph and research-taste excerpts
Primary source: https://spinningup.openai.com/en/latest/spinningup/spinningup.html Author: Joshua Achiam, OpenAI Date: October 13th, 2018 Related local cache: docs/evidence/spinningup_researcher.md Fetch-status: index only. The full page text now lives in docs/evidence/spinningup_researcher.md (fetched 2026-08-15); this file keeps the source graph so the two caches do not hold the same 3.3k words twice (CLAUDE agent) Use: RL research-process evidence, especially for source graph, fair comparisons, seeds, preregistration, and ablations.
Why this matters for agents
Spinning Up is not just an RL textbook page. Its researcher page is a compact research apprenticeship guide. It points to the same battle-tested debugging and reproducibility references already cached in this repo, then adds project selection and rigorous comparison advice.
Quotes
The quotes that used to sit here (simplicity is critical, iterate fast in simple environments, avoid reinventing the wheel, handicap the baseline, seeds, preregistration, ablations) are all in the full page text at spinningup_researcher.md. Read that file for the wording; this one only carries the source graph. (CLAUDE agent, 2026-08-15)
Source graph
Spinning Up intro references, with local status:
- Alex Irpan, Deep Reinforcement Learning Doesn't Work Yet: https://www.alexirpan.com/2018/02/14/rl-hard.html. Local cache: docs/evidence/alexirpan_rl_hard.md.
- Islam et al., Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control: https://arxiv.org/pdf/1708.04133. Not separately cached; discussed/cited inside Henderson local cache.
- Henderson et al., Deep Reinforcement Learning that Matters: https://arxiv.org/pdf/1709.06560. Local cache: docs/evidence/henderson_2018_deep_rl_matters.md.
- Matthew Rahtz, Lessons Learned Reproducing a Deep RL Paper: http://amid.fish/reproducing-deep-rl. Local cache: docs/evidence/amid_fish_reproducing_deep_rl.md.
- David Silver UCL RL course: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html. Not cached.
- Berkeley Deep RL course: http://rll.berkeley.edu/deeprlcourse/. Not cached.
- Deep RL Bootcamp lectures: https://sites.google.com/view/deep-rl-bootcamp/lectures. Reddit index cache: docs/evidence/reddit_deeprl_bootcamp_2017_75m5vd.md.
- John Schulman, Nuts and Bolts of Deep RL: http://joschu.net/docs/nuts-and-bolts.pdf. Local cache: docs/evidence/joschu_nuts_and_bolts.md.
- Tim Rocktaschel et al., Advice for Short-term Machine Learning Research Projects: https://rockt.github.io/2018/08/29/msc-advice.html. Not cached.
- Catherine Olsson / 80,000 Hours, ML Engineering for AI Safety & Robustness: https://80000hours.org/articles/ml-engineering-career-transition-guide/. Not cached.
Likely follow-up cache candidates
Priority 1:
- Chris Olah, research taste: short and directly named by Nanda.
- Jacob Steinhardt, Research as a Stochastic Decision Process: directly named by Nanda for prioritization.
- Tim Rocktaschel et al., short-term ML research projects: directly named by Spinning Up for research growth.
Priority 2:
- David Silver/UCL, Berkeley Deep RL, Deep RL Bootcamp: curriculum material, less directly research-taste except via RL mastery.
- Catherine Olsson/80k: career/field-entry framing; useful if the skill expands beyond project-level research taste.