TL;DR version
Stop at any step if it doesn’t spark joy
Read title
Read abstract very carefully
Skip intro and skim methods
Figure 1 and Table 1 (review eligibility from methods as needed)
Main result tables and figures (refer to methods again for analytics)
Read text of results
Read the supplementary appendix for other results or methods details
Read authors names, affiliations, funding body
(Make up your mind about paper)
Limitations section of discussion
Read rest if still puzzled about authors intentions or thoughts
Sek recently asked if anyone reads the full paper. See some interesting replies in thread:
What follows is the technique I have slowly developed to read a paper. It’s idiosyncratic, and has evolved over time. This guide is more about the hierarchy of the order I read a paper rather than how to understand or critique a study. It makes for efficiency more than anything else. At any step if the study doesn’t interest me anymore I stop. This order minimizes wasting time. YMMV and all the other caveats.
Read title
Seems obvious, but useful when you are perusing table of contents, or an RSS feed and want to decide to click ‘more’ or ‘read abstract’.
Read abstract carefully
Often derided as ‘did you read beyond the abstract’. Doesn’t mean you should skip it. The abstract’s going to provide you with a snapshot of the study, including study design, population, and main result. The structured abstract is very useful - hopefully more journals will adopt it.
Skip the introduction and skim the methods
You have decided now that you are going to go beyond the abstract. You have made the effort to get the full text. Go straight to methods. Skim so that you know what is where in the methods - you already know the study design and population at a very superficial level. Don’t get bogged down here now.
Figure 1 and Table 1
For all clinical research papers there *must* be a figure 1 which explains how the participants (or studies if a systematic review) were assembled. Sometimes this can even be buried in the supplementary appendix - find and it and look at it carefully. How many screened, how many eligible, how many consented/randomized? How many were lost to follow up and how many were included in the different analysis? This is also the time you go back to methods to check out the eligibility, time, and place of study. Sometimes there is no good figure 1 (and for #nephjc we have sometimes made our own versions to make sense). Especially in observational studies it is useful to pay attention to how the cohort was assembled. Is it propensity score matched? See unmatched and matched groups. How was exposure defined and does that make sense from the figures? In systematic reviews, do the number of studies found from the databases make sense? Figure 1 is very important. Play close attention to figure 1. Did I say you should understand figure 1 closely?
Next up is table 1. If it is an RCT, don’t look for imbalances or p values between groups. It is a wrong and unscientific practice. Look at table 1 to see what kind of population is assembled. Are these the kind of patients you see? Would you be comfortable extrapolating the results from this population to your patient? If its an observational study - do pay attention to the group differences. Let’s say age is different - the subsequent adjusted analysis should be adjusted for age, but try to understand how any differences in covariates can be explained. Could they signal an underlying selection bias? Again at this step - go back to methods to see eligibility criteria.
Main Results (tables and figures)
Table 2 and later, and figure 2 and later should be about the rest of the results. Read them, go back to methods to read outcomes and analytic methods.
Read text of results
Read the actual text after you have checked out all the tables and figures. See if your assessment agrees with how they state the results.
Supplementary appendix, protocol
Check out what is provided in the supplementary appendix. JAMA and NEJM now provide the protocol for every RCT published as a supplement. Sometimes the protocol will be published as a separate paper. Check out the registration details.
The supplement may also have additional analysis, details about analytic methods, changes in protocol and myriad other details.
Authors, affiliations and funding
Self explanatory. Pay attention to where the authors are from and where the study was done. Who funded the study and what was their involvement?
Make up your mind
By this stage you should know what you think about the paper. The headline results, the weaknesses. Pause and think about these aspects before (if) you read the discussion.
Limitations section
Jump to the limitations and see if the authors discuss the limitations you picked up. Do they mention some weaknesses you had not considered? Every paper is an educational opportunity.
The rest of the paper
Are you presenting at a journal club? Reading a paper because you have to peer review it? Puzzled about some of the decisions or the findings? Only then read the introduction and the rest of the discussion. These are a mix of background review of topic, fiction and spin. That’s the authors’ version of the story. They are often useful, but reading them before you read the methods and results can bias you. We like stories, we like to believe narratives. Don’t let that happen without a fight.
That’s it. 10 steps to read a paper. Needless to say, this applies to original clinical research, not reviews or commentaries, and not to basic science papers. Feel free to trash or suggest alternatives.