Image: Marcus Gossler, CC BY-SA 3.0How to read an economics paper as a beginner
Every unit of the Economics Masterclass presents one published paper, and every one of those papers is free to read. Most of them are from the Journal of Economic Perspectives, which the American Economic Association publishes precisely so that economists can read outside their specialism, and which is the best place for a beginner to start. Even so, a paper is a different kind of text from a textbook, and it rewards being read in a different order. Here is the order, with one article as the example: Joshua Angrist and Jörn-Steffen Pischke's The Credibility Revolution in Empirical Economics, from 2010, which is the research spotlight of Unit 1.
Start with the abstract, and read it twice
The abstract is the paper in 150 words. Read it once to get the subject and once to find the claim, which is usually a single sentence that begins with something like we find or we show. Angrist and Pischke's abstract says that empirical economics has become more credible because of better research design, that the change came from the use of experiments and quasi-experiments, and that the criticism that such designs answer narrow questions is misplaced. That is the whole argument. Everything that follows is evidence and elaboration, and you now know what to look for.
Then the introduction, which is the paper again
Economics papers have long introductions, often five or six pages, and they are written to be read on their own. A good introduction states the question, says why it is hard, explains what the authors did, reports the main results with numbers, and says how the results relate to what was known. If you read only the abstract and the introduction of a paper you have understood most of what the authors want you to understand. Angrist and Pischke open with a 1983 article by Edward Leamer that accused econometrics of being a con, with results that depended on which variables the researcher happened to include, and the introduction tells you that the rest of the paper is an answer to Leamer twenty-seven years later.
Skip to the conclusion
Before reading the body, read the last section. Authors are more candid at the end: they say what the results do not show, what they would do with better data, and which objections they take seriously. In an empirical paper the conclusion often contains a sentence about external validity, which is whether a result found in one place and time holds elsewhere, and that sentence is worth more than the whole of the robustness section.
Now the body, with three questions
The body of an empirical paper has a standard shape: data, method, results, robustness. Read it with three questions in mind. First, what is being compared with what? Every causal claim in economics rests on a comparison between a group that received something and a group that did not, and the whole credibility of the paper rests on whether the second group is a fair stand-in for what would have happened to the first. Angrist and Pischke's examples are the comparisons that work: New Jersey and Pennsylvania in the minimum wage study of Card and Krueger, lottery winners and losers in school admissions, people born just before and just after a cutoff date. When you find the comparison, ask what would have to be true for it to be fair, and whether the authors have checked that it is. Economists call this the identification strategy, and in a modern paper it has a section of its own.
Second, how large is the effect, and in what units? A result that is statistically significant can be tiny, and a table reports both. Find the main number, find its units, and translate it into something you can picture: a rise of 0.03 log points in earnings is a rise of about 3 per cent, which for a worker on 30,000 is 900 a year. Third, how uncertain is it? Every estimate in a table comes with a standard error in brackets beneath it, and the honest range for the true effect is roughly the estimate plus or minus twice that number. If the range includes zero, the paper has not found an effect; if it runs from a trivial value to a large one, the paper has found something but not how much. The temptation is to read the estimate and skip the bracket, and most of the overclaiming in economics journalism comes from doing so.
The tables
Tables are where the paper actually is. Each column is usually one version of the same estimate with a different set of controls or a different sample, and the first column is the simplest. Read the first column and the last, and look at whether the main number moves between them. If it hardly moves as controls are added, the comparison is probably fair; if it moves a lot, something that was not controlled for in the first column mattered, and you should wonder what else might. Then read the notes under the table, which say what the sample is, what the units are, and what the stars mean; one star is usually significance at the 10 per cent level, two at 5, three at 1.
Where the uncertainty hides
Standard errors capture sampling uncertainty, the chance that a different sample would have given a different number. They do not capture the uncertainty about whether the comparison was fair, whether the data measure what they claim, or whether the result would hold in another country or decade, and those are usually larger. A paper that reports an effect in one city in one year to three decimal places has told you about that city and that year. The credibility revolution Angrist and Pischke describe made the first kind of uncertainty smaller and the comparisons more honest; the remaining kinds are what the discussion sections of papers, and the replication studies that follow them, are about. Unit 1 of the masterclass makes this point with Card and Krueger, and the post on minimum wages follows the thirty years of argument that one careful comparison started.
A worked number
Suppose a table reports that a training programme raised participants' earnings by 1,200 dollars a year with a standard error of 500. The estimate is 2.4 standard errors from zero, so it carries two stars, and the plausible range is about 200 to 2,200 dollars. If the programme cost 3,000 dollars per participant and the gain lasts five years, the return ranges from well below cost to more than three times cost. The paper has shown that the programme works and has not shown whether it is worth paying for, which is a different question that needs a different table, and noticing the difference is most of what reading a paper well consists of.
A reading plan
Take one paper from the masterclass each week, the one in the unit you are on, and spend forty minutes on it in the order above: abstract twice, introduction, conclusion, then the body with the three questions and the tables. Write five sentences afterwards: what the question was, what was compared with what, how big the effect was, how uncertain, and whether you believe it. After sixteen weeks you will read the economics pages of a newspaper differently, and you will have the habit that distinguishes an economist from a person who has read some economics.


