wassname
2.3 KiB
Research Taste Exercises - Chris Olah (2021-01-09)
Source: https://colah.github.io/notes/taste/ Author: Chris Olah Date: Posted Jan 9, 2021 Fetch-status: excerpted from HTML via browser. Use: direct source for research-taste training exercises; cited by Nanda's shared draft and public taste post.
Why this matters for agents
Olah gives concrete exercises for getting more feedback on taste without spending months executing every idea. This is a good reference for an agent asked to help a researcher improve project selection or calibrate idea quality.
Quotes
One of the most important aspects of growing as a researcher is developing research taste -- roughly, the ability to chose good problems to work on.
I think the fundamental issue is that actually testing whether a research idea you come up with is good is very expensive. Often it takes months, so you only really get a few pieces of feedback on your taste every year.
Many of the following exercises are really strategies for getting (proxy) feedback on more research ideas faster.
Write down a list of research ideas. Have a mentor you respect rate each idea 1-10. Discuss ideas where you disagree with them after reflection.
Pay attention when other people try ideas you’ve had. How did the results compare with your expectations?
Interview researchers around you on their taste. Why do they work on the problems they do? How do they pick problems? What’s their “big picture” of research?
Critically consider your research taste, and the community taste around you. Your taste is likely very influenced by your research cluster (your collaborators, advisor, etc).
Failure Mode 1: Getting overly attached to one research direction / falling into sunk costs.
Failure mode 2: Lack of research knowledge / intimacy.
Theoretical knowledge is table stakes for research taste. You can’t have research taste in a vacuum.
Failure mode 3: Environment not aligned with your interests.
Source graph
Links visible in the post worth follow-up:
- Hamming, You and Your Research: linked via YouTube.
- Michael Nielsen, Principles of Effective Research: https://michaelnielsen.org/blog/principles-of-effective-research/
- Andy Matuschak taste-related thread: linked as Twitter, may need archival route.