wassname
3.6 KiB
How I Think About My Research Process: Explore, Understand, Distill - Neel Nanda (2025-04-26)
Source: https://www.lesswrong.com/posts/hjMy4ZxS5ogA9cTYK/how-i-think-about-my-research-process-explore-understand Mirror/sequence URL visible on page: https://www.lesswrong.com/s/5GT3yoYM9gRmMEKqL/p/hjMy4ZxS5ogA9cTYK Author: Neel Nanda Date: 26th Apr 2025 Fetch-status: excerpted from LessWrong HTML via browser plus cross-checked against local shared draft. Use: research-process / research-taste evidence, especially for agents deciding what mode of work they are in.
Why this matters for agents
Nanda frames empirical research as stage-dependent. A model should not demand a crisp hypothesis when the right stage is exploration; it should not accept weak, cherry-picked evidence when the task has moved into understanding or distillation.
Quotes
This guide focuses more on the strategic (high-level direction, when to give up or pivot, etc) and tactical (what to do next, how to prioritise, etc) aspects of research - the "how to think about it" rather than just the "how to do it." Some of skills (coding, reading papers, understanding ML/mech interp concepts) are vital for how to do it, but not in scope here.
How to get started? Strategic and tactical thinking are hard skills, and it is rare to be any good at them when starting out at research (or ever tbh). The best way to learn them is by trying things, making predictions, seeing what you get right or wrong (i.e., getting feedback from reality), and iterating.
I see research as breaking down into a few stages:
- Ideation - Choose a problem/domain to focus on
- Exploration - Gain Surface area
- North star: Gain information
- Understanding - Test Hypotheses
- North star: Convince yourself of a key hypothesis
- Distillation - Compress, Refine, Communicate
- North star: Compress your research findings into concise, rigorous truth that you can communicate to the world
At the start, your understanding of the problem is often vague. Naively, it’s easy to think of research as being about testing specific hypotheses, but in practice you often start out not even knowing the right questions to ask, or the most promising directions. The exploration stage is about moving past this.
Not having a clear goal/next step doesn’t mean that you don’t need to prioritise! Prioritise for information gain.
Frequently ask yourself “am I getting enough information per unit time?” If you haven’t learned anything recently, shake it up.
The mark of a good researcher is a deep commitment to skepticism of your results.
A great experiment elegantly, and conclusively distinguishes between several plausible hypotheses, validates non-trivial predictions made by one hypothesis, and is tractable to implement in practice.
The north star here is to distill your research findings into concise, rigorous truth that you can communicate to the world.
Write to inform, not persuade - if you are clear (a high bar), and your results are interesting, people will likely appreciate your work.
Source graph
High-value links inside or adjacent to this post:
- ARENA curriculum: https://arena-chapter1-transformer-interp.streamlit.app/
- Nanda paper reading list: https://www.alignmentforum.org/posts/NfFST5Mio7BCAQHPA/an-extremely-opinionated-annotated-list-of-my-favourite
- Chris Olah, research taste: https://colah.github.io/notes/taste/
- Nanda Othello research process write-up: https://www.alignmentforum.org/s/nhGNHyJHbrofpPbRG/p/TAz44Lb9n9yf52pv8
- Nanda standards post: https://www.neelnanda.io/blog/35-standards