Critical reading of articles for medical students

The professor projects the summary of a clinical trial: "the risk of hospitalization fell by almost half." The class takes notes. No one asks how many patients were studied, who they were, how long they were followed or who funded the study. On the exam, the sentence comes back as truth. In practice, years later, the same student will receive news about a "revolutionary new treatment" and will need to decide whether to believe it.

NEXO is a daily medical diagnosis game: one clinical case a day, six progressive clues, and ICD-10 proximity feedback across five levels.

Critical reading is the ability to assess whether a study answers well the question it proposes and whether the results apply to the patient in front of you. This guide presents a three-question script, explains study designs and the most common biases, shows what studies say about how to learn critical reading and offers a journal club format for medical school.

Key points

• Critical reading answers three questions: is the study valid, what are the results and do they apply to my patient.
• Each study design has typical weaknesses, and knowing them speeds up reading.
• Reviews show that several teaching formats improve knowledge and attitudes, with no clearly superior method.
• A randomized trial with final-year students found gains in knowledge and confidence after a short training.
• Journal clubs work best with regular meetings, a clear goal, trained leadership and appraisal scripts.

Why read critically

The volume of information

Thousands of health articles are published every day. Guidelines, classes and summaries filter part of this volume, but they also carry choices and limitations. Those who know how to read an article can check where a recommendation comes from and how much to trust it.

A study result is not ready-made truth

A statistically significant result may be too small to matter, may come from a population very different from yours or may depend on an outcome that does not interest the patient. Critical reading separates what the study showed from what the summary or the headline promises.

The abstract is the most read and the most optimistic part of an article. It has limited space, and the authors choose what to highlight. That is why critical reading always goes through the methods and the results tables, where loss to follow-up, secondary outcomes and adverse effects that did not fit in the abstract appear.

The students' starting point

In a study with 417 third- and fourth-year students from 40 American medical schools, the mean score on a validated evidence-based medicine test was 47.2% (Smith et al., 2016). Questions on internal validity, effect magnitude and confidence intervals had correct rates of 43.6%, 37.8% and 34.3%, respectively (Smith et al., 2016). That is, even close to graduation, the central points of critical reading tend to be the weakest.

The link with clinical reasoning

Interpreting tests, with sensitivity, specificity and pre-test probability, is part of the same family of skills and has its own guide on the blog, about pre-test probability. Here the focus is study design and risk of bias: before using a number, you need to know whether it deserves trust.

A three-question script

1. Is the study valid?

The first question is about method. Is the study question clear? Does the chosen design answer it? Were the compared groups similar at the start? Were there important losses to follow-up? Did those who assessed the outcomes know which group each participant belonged to?

2. What are the results?

The second question is about the size and precision of the effect. Look at the absolute effect, beyond the relative one: a large relative risk reduction may correspond to a small absolute difference when the event is rare. Also look at the confidence interval, which shows the range of values compatible with the data.

3. Do the results apply to my patient?

The third question is about applicability. Do the participants resemble your patient in age, severity and context? Does the measured outcome matter to them? Do the benefits outweigh the risks, the burdens and access to the intervention?

An example of quick reading

Imagine a trial that compares a new antihypertensive drug with placebo in adults without other diseases, followed for six months, with office blood pressure as the primary outcome. Critical reading asks: why placebo, and not usual treatment? Are six months enough to assess adverse effects? Office pressure is a surrogate outcome; does the study show anything about myocardial infarction or stroke? And would your patient, older and with kidney disease, have been among the participants?

Study designs and their weaknesses

Randomized clinical trial

It is the strongest design for assessing interventions, because randomization tends to balance known and unknown factors between the groups. Typical weaknesses: poorly done allocation concealment, lack of blinding when it would have been possible, unequal losses to follow-up, analysis that excludes those who abandoned treatment, and surrogate outcomes.

Cohort

It follows people exposed and not exposed over time. It serves for risk factors and prognosis and for rare or late effects. The main weakness is confounding: the groups may differ in factors that also influence the outcome.

Case-control

It starts from the outcome and looks back in search of exposures. It is efficient for rare diseases, but vulnerable to recall bias and to inadequate choice of controls.

Cross-sectional

It measures exposure and outcome at the same moment. It is useful for estimating prevalence, but rarely allows knowing what came first.

Systematic review and meta-analysis

It gathers studies on the same question with explicit search and criteria. Quality depends on the search, on the assessment of risk of bias of the included studies and on the similarity between them. A meta-analysis of bad studies remains a synthesis of bad studies.

Biases every student should recognize

Selection bias

It happens when the participants, or the compared groups, differ systematically. Example: a study of a surgical treatment that includes only patients considered fit for surgery and compares them with those who were not operated on.

Measurement bias

It happens when the outcome is measured differently between the groups. Assessors who know who received the treatment may, without meaning to, record greater improvements in that group.

Confounding

A third factor, linked to the exposure and to the outcome, creates an apparent association. Someone who takes a supplement may also exercise more and smoke less, and the effect attributed to the supplement may come from these habits.

Publication bias

Studies with positive results are more likely to be published. The result is a literature that overestimates effects. The topic reappears in this series' guide on conferences, because many abstracts presented never became an article.

Surrogate outcomes

Laboratory and imaging markers respond faster than clinical events and are therefore attractive. Improving a marker, however, does not always improve the patient's life.

How to learn critical reading

What the reviews show

A systematic review on teaching evidence-based medicine in undergraduate courses included 27 studies (Ahmadi et al., 2015). Some strategies improved knowledge, attitudes and skills, none proved superior to the others and there was no evidence of transfer to clinical practice (Ahmadi et al., 2015). Good-quality distance learning was as effective as in-person learning (Ahmadi et al., 2015).

A Cochrane review on teaching critical reading to health professionals, without including students, found only three eligible studies, with 272 participants (Horsley et al., 2011). The authors concluded that low-intensity interventions may bring modest gains and called for better trials (Horsley et al., 2011).

A trial with students

In a randomized trial with 256 sixth-year students in Iran, the group that received multimedia material and an online critical reading workshop performed better in all the dimensions assessed (Sasannia et al., 2022). The effects were large in knowledge and confidence and moderate in attitude and appraisal skill (Sasannia et al., 2022).

The study comes from a single school and measures the effect right after the training. Even so, it shows that a short investment already moves knowledge and confidence.

Journal clubs

A systematic review of 15 studies on journal clubs in undergraduate education concluded that they are effective and well accepted for teaching evidence-based medicine (Bello and Grant, 2023). Sessions longer than two hours reduced satisfaction, classes linked to the club improved knowledge gain and retention, and critical appraisal scripts helped with writing and attitude (Bello and Grant, 2023).

Another review on journal clubs in health identified regular and scheduled meetings, a clear objective, a suitable time, a trained leader who chooses the articles and leads the discussion, and prior sending of the material as features of success (Deenadayalan et al., 2008). No study in that review assessed transfer to clinical practice (Deenadayalan et al., 2008).

A journal club format

• Frequency: one meeting every two or three weeks, on a fixed date.
• Duration: up to one and a half hours.
• Preparation: article and appraisal script sent one week in advance.
• Leadership: a student presents, and a professor or resident guides the discussion.
• Structure: the three questions of the script, in that order, and a final decision about the clinical case that motivated the choice of article.
• Record: a half-page summary kept in a file of the league or the class.

Where to start on your own

Choose an article cited in a recent class. Read the abstract first and write down what you expect to find. Then read methods and results with the three-question script and compare with what the abstract promised. Repeat with articles of different designs. For those doing undergraduate research, this practice shortens the path a lot; the topic has its own guide in this series.

From the article to the case in NEXO

Critical reading ends with a clinical question: does this apply to my patient? The NEXO case of the day trains the other half of that skill, building hypotheses from the patient's information. Each case is fictional, with a real clinical presentation, and the diagnosis arrives in up to six clues. Putting a well-read article together with well-organized reasoning is what takes evidence to the decision.

The guide what NEXO is explains the whole game, and the release of modes is on the NEXO economy page. To take evidence to the decision, the clinical decision-making guide complements this one.

If you coordinate a medical school, a league or a residency and want cases to open journal clubs, see NEXO Closer. Researchers in medical education can propose studies through NEXO Researcher. Doctors who review cases take part through NEXO Curator, and those who create health content on social media can take part through NEXO Creators.

Frequently asked questions

What is critical reading of articles?

It is assessing whether a study is valid, what the results are and whether they apply to the patient in question.

Where should I start?

With the three-question script, applied to articles cited in classes, of different designs.

What is the best method to learn?

Several strategies improve knowledge and attitudes, with no clearly superior one (Ahmadi et al., 2015).

Do journal clubs work?

A review of 15 studies in undergraduate education concluded that they do, for teaching evidence-based medicine (Bello and Grant, 2023).

Does a short training make a difference?

In a randomized trial, multimedia material and an online workshop improved the knowledge and confidence of final-year students (Sasannia et al., 2022).

What is the difference from interpreting tests?

Interpreting tests uses sensitivity, specificity and pre-test probability; critical reading assesses the study design and risk of bias.

Critical reading checklist

1. Identify the question and the study design.
2. Check the groups at the start.
3. Look for losses to follow-up.
4. See who assessed the outcomes.
5. Look at the absolute effect and the confidence interval.
6. Ask whether the outcome matters to the patient.
7. Decide whether it applies to your patient.

Want to train the reasoning that comes after reading? Play the case of the day in NEXO.

References

1. Ahmadi SF, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: a BEME systematic review. Med Teach. 2015. https://pubmed.ncbi.nlm.nih.gov/25401408/
2. Horsley T, Hyde C, Santesso N, Parkes J, Milne R, Stewart R. Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev. 2011. https://pubmed.ncbi.nlm.nih.gov/22071800/
3. Sasannia S, Amini M, Moosavi M, Askarinejad A, Moghadami M, Ziaee H, et al. Critical appraisal skills training to undergraduate medical students: A Randomized Control Study. J Adv Med Educ Prof. 2022. https://pubmed.ncbi.nlm.nih.gov/36310666/
4. Bello JO, Grant P. A systematic review of the effectiveness of journal clubs in undergraduate medicine. Can Med Educ J. 2023. https://pubmed.ncbi.nlm.nih.gov/37719396/
5. Deenadayalan Y, Grimmer-Somers K, Prior M, Kumar S. How to run an effective journal club: a systematic review. J Eval Clin Pract. 2008. https://pubmed.ncbi.nlm.nih.gov/19018924/
6. Smith AB, Semler L, Rehman EA, Haddad ZG, Ahmadzadeh KL, Crellin SJ, et al. A Cross-Sectional Study of Medical Student Knowledge of Evidence-Based Medicine as Measured by the Fresno Test of Evidence-Based Medicine. J Emerg Med. 2016. https://pubmed.ncbi.nlm.nih.gov/26961178/