wassname
3.7 KiB
Highly Opinionated Advice on How to Write ML Papers - Neel Nanda (2025-05-12)
Source: https://www.lesswrong.com/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers Author: Neel Nanda Date: 12th May 2025 Fetch-status: excerpted from LessWrong HTML via browser. Use: distillation and paper-writing evidence. This is adjacent to the research-process sequence, and directly useful when turning messy findings into a public artifact.
Why this matters for agents
This post is the operational version of the distillation stage: compress the research into a few claims, red-team the evidence, write to inform rather than persuade, and spend disproportionate care on the abstract, intro, figures, and limitations.
Quotes
The essence of an ideal paper is the narrative: a short, rigorous and evidence-based technical story you tell, with a takeaway the readers care about.
The first step is to compress your research into these claims.
Experimental Evidence: This is absolutely crucial to get right and aggressively red-team, it’s how you resist the temptation of elegant but false narratives.
Inform, not persuade: Avoid the trap of overclaiming or ignoring limitations.
Your research only matters if people read, understand, and build upon it.
At its core, a paper should present a narrative of one to three specific concrete claims that you believe to be true, that build to some useful takeaway(s).
Readers will rarely take away more than a few sentences of content. Choose those sentences carefully.
Generally, stronger statements make for more interesting papers, but require higher standards of evidence - resist the temptation to overclaim for clicks!
Warning: Before moving into paper-writing mode, it's crucial to verify that your evidence is actually correct.
Novelty means it expands our knowledge.
Rigorous, at-scale replications of shaky results, negative results of seemingly promising hypotheses, and high-quality failed replications of popular papers are all very valuable contributions.
A particularly important thing to get right is extensive red-teaming: you should spend a good amount of your time, both during the original research and now, red teaming your narrative.
Good experiments distinguish between hypotheses.
This skepticism and sanity checking is especially key for particularly surprising or novel bits of evidence.
Ablation studies: When a paper introduces a complex new method, there are often several moving parts.
Track pre/post-hoc analysis.
Quality Over Quantity: Try to prioritise having at least one really compelling and hard to deny experiment, over a bunch of mediocre ones.
Baselines are Crucial.
The subtlety of baselines: It's not enough to just have them; you must strive to have the strongest possible baselines.
The Guiding Question for Evidence: Ultimately, the question to ask about your evidence is: "Should this update a reader's beliefs about my claims?"
Reproducibility & Publishing code: Rigour can be in the eye of the beholder: if readers cannot understand or verify it for themselves, it’s far harder to consider it rigorous.
A key challenge in paper writing is the illusion of transparency - you have spent months steeped in the context of this research project.
Source graph
Links visible in this post worth follow-up:
- Research process sequence: https://www.lesswrong.com/s/5GT3yoYM9gRmMEKqL
- Jakob Foerster writing advice: https://www.jakobfoerster.com/
- Jacob Steinhardt writing/research advice: https://cs.stanford.edu/~jsteinhardt/
- Refusal is mediated by a single direction: https://arxiv.org/abs/2406.11717
- Nanda grokking work: https://arxiv.org/abs/2301.05217
- Paper writing checklist: Google Docs link visible in post, not cached.