mirror of
https://github.com/wassname/ml_debug.git
synced 2026-06-27 01:00:14 +08:00
wassname
43 lines
2.3 KiB
Markdown
43 lines
2.3 KiB
Markdown
# Research Taste Exercises - Chris Olah (2021-01-09)
|
||
|
||
Source: https://colah.github.io/notes/taste/
|
||
Author: Chris Olah
|
||
Date: Posted Jan 9, 2021
|
||
Fetch-status: excerpted from HTML via browser.
|
||
Use: direct source for research-taste training exercises; cited by Nanda's shared draft and public taste post.
|
||
|
||
## Why this matters for agents
|
||
|
||
Olah gives concrete exercises for getting more feedback on taste without spending months executing every idea. This is a good reference for an agent asked to help a researcher improve project selection or calibrate idea quality.
|
||
|
||
## Quotes
|
||
|
||
> One of the most important aspects of growing as a researcher is developing research taste -- roughly, the ability to chose good problems to work on.
|
||
|
||
> I think the fundamental issue is that actually testing whether a research idea you come up with is good is very expensive. Often it takes months, so you only really get a few pieces of feedback on your taste every year.
|
||
|
||
> Many of the following exercises are really strategies for getting (proxy) feedback on more research ideas faster.
|
||
|
||
> Write down a list of research ideas. Have a mentor you respect rate each idea 1-10. Discuss ideas where you disagree with them after reflection.
|
||
|
||
> Pay attention when other people try ideas you’ve had. How did the results compare with your expectations?
|
||
|
||
> Interview researchers around you on their taste. Why do they work on the problems they do? How do they pick problems? What’s their “big picture” of research?
|
||
|
||
> Critically consider your research taste, and the community taste around you. Your taste is likely very influenced by your research cluster (your collaborators, advisor, etc).
|
||
|
||
> Failure Mode 1: Getting overly attached to one research direction / falling into sunk costs.
|
||
|
||
> Failure mode 2: Lack of research knowledge / intimacy.
|
||
|
||
> Theoretical knowledge is table stakes for research taste. You can’t have research taste in a vacuum.
|
||
|
||
> Failure mode 3: Environment not aligned with your interests.
|
||
|
||
## Source graph
|
||
|
||
Links visible in the post worth follow-up:
|
||
- Hamming, You and Your Research: linked via YouTube.
|
||
- Michael Nielsen, Principles of Effective Research: https://michaelnielsen.org/blog/principles-of-effective-research/
|
||
- Andy Matuschak taste-related thread: linked as Twitter, may need archival route.
|