Step 2 CK · Ethics, Biostatistics & Patient Safety

Evaluating Drug Advertisements & Research Abstracts

Name: Evaluating Drug Advertisements & Research Abstracts
Creator: Veli Bakalov, MD

Put every marketing claim on trial against the numbers printed on the page

Ethics, Biostatistics & Patient Safety·Biostatistics·5 min read·9 high-yield points·Reviewed by Veli Bakalov, MD @HemeOncBuddy

Study materialsNo account needed

Card deckActive-recall cards · quick checks · quiz

High-yield

Start every ad or abstract by identifying the design, population, intervention, comparator, and prespecified primary outcome.
A claim is supported only if the study measured that outcome in that population at that dose; claims from unmeasured outcomes, failed-primary secondary endpoints, or subgroups are unsupported.
ARR = control event rate − treatment event rate; RRR = ARR ÷ control rate; NNT = 1/ARR, stated with its outcome and time frame.
NNH = 1/ARI, where ARI is the adverse-event rate on the drug minus the rate on the comparator.
Relative risk reductions inflate small absolute benefits: 2% to 1% is a 50% RRR but only a 1% ARR (NNT 100).
A placebo-controlled trial cannot support "better than drug X," and an underdosed active comparator manufactures superiority.
If the patient in the stem meets an exclusion criterion (elderly, renal or hepatic impairment, pregnancy), the results do not apply to that patient.
Industry sponsorship, short follow-up, and surrogate outcomes warrant skepticism; zero events in n patients still allows a true rate up to about 3/n.
A statistically significant result is not clinically meaningful if the effect falls below the minimal clinically important difference.

A routine for advertisement and abstract items

The format. Some USMLE items present a pharmaceutical advertisement or a research abstract and then ask whether its claims are justified, what a number means, or whether the results apply to a particular patient. The product may be unfamiliar, so the answer must come from what is printed on the page, including tables, footnotes, and fine print, which often hold the exclusion criteria, dosing, and funding source. Read the question first, then search the stimulus for the specific number or criterion it targets.

DesignRCT, cohort, or case-control; blinded or open-label; length of follow-up.
PopulationInclusion and exclusion criteria.
Intervention and comparatorDrug, dose, and schedule; placebo or active comparator, and the comparator's dose.
Primary outcomeThe prespecified endpoint and its absolute event rate in each arm.
Test each claimSupported only if the data shown measured it, in this population, at this dose.

Testing claims against the data

Treat each sentence of an advertisement as a hypothesis: it stands only when an outcome measured in the trial backs it directly.

Common unsupported claims

Claim pattern	Why it fails
Outcome never measured ("prevents heart attacks" from an LDL-only trial)	No data on the claimed outcome
Benefit taken from a secondary endpoint after the primary failed	Hypothesis-generating only
Subgroup claim ("especially effective in women")	Post hoc subgroups are often chance findings
"Better than drug X" from a placebo-controlled trial	No head-to-head comparison
Causal claim from a cohort or case-control study	Shows association; confounding remains possible
"For all patients" or a dose, duration, or population not studied	Results cannot be extrapolated beyond the trial

Doing the math: absolute benefit and harm

Definitions. With CER as the control event rate and EER as the experimental event rate, the absolute risk reduction is ARR = CER − EER, the relative risk reduction is RRR = (CER − EER)/CER, and the number needed to treat is NNT = 1/ARR. For harms, the absolute risk increase is ARI = adverse-event rate on the drug − rate on the comparator, and the number needed to harm is NNH = 1/ARI. Adverse-effect rows are often buried in the fine print.

Worked example: new antiplatelet drug vs clopidogrel, 12 months

Outcome	Clopidogrel	New drug	Absolute difference	Number needed
CV death, MI, or stroke	8.0%	6.0%	ARR 2.0%	NNT 50
Major bleeding	1.5%	2.5%	ARI 1.0%	NNH 100

The advertised "25% reduction" is the RRR (2.0 ÷ 8.0).

Why relative framing flatters. Cutting an event rate from 2% to 1% is a "50% reduction" but an ARR of only 1% (NNT 100). Relative reductions are often similar across risk groups, whereas the ARR shrinks as baseline risk falls, so the same drug is worth far less to a low-risk patient. Advertisements favor relative numbers for exactly this reason.

Using NNT and NNH. An NNT has meaning only with its outcome and time frame attached (treat 50 patients for 3 years to prevent one stroke). Round the NNT up to the next whole number (1/0.03 = 33.3, so 34). Compare NNT with NNH for outcomes of similar severity: an NNT of 50 to prevent one stroke weighed against an NNH of 25 to cause one major bleed is a poor trade.

Comparators, doses, patients, and credibility

Comparators. Beating placebo says nothing about beating existing therapy, so a "better than" claim requires a head-to-head trial. In a head-to-head trial, confirm that the active comparator was given at its standard dose, route, and schedule; pitting a new drug's maximum dose against the lowest dose of an established one (for example, atorvastatin 10 mg) manufactures superiority.

Doses. Efficacy at one dose, formulation, or duration does not transfer to another. Benefit shown at 20 mg twice daily cannot support a promoted 10 mg once-daily regimen, which may look safer but was never shown to work, and safety data from a 12-week trial cannot support years of use.

The patient in the stem. Trials commonly exclude older adults, patients with renal or hepatic impairment, pregnant or breastfeeding patients, children, and those with serious comorbidity or interacting drugs. If the stem's patient meets an exclusion criterion, the trial provides no evidence about benefit or harm for that patient, however well it was run, and a "for all patients" claim is unsupported.

Sponsorship and follow-up. Industry funding and authors' financial ties do not invalidate a study, but they raise the risk of favorable comparators, convenient endpoints, and spin, so read the funding and conflict-of-interest disclosures. Short follow-up misses delayed and rare adverse effects.

3/nrule of three: with zero events among n patients, the true rate may still be as high as about 3/n (0.5% when n = 600)

Surrogate outcomes. Lower LDL, higher bone density, lower HbA1c, or tumor shrinkage does not establish fewer myocardial infarctions, fractures, complications, or deaths; a claim of clinical benefit needs a clinical outcome. A quick test: would a patient notice the outcome without a laboratory test or scan?

Statistical vs clinical significance. A very large trial can make a trivial difference statistically significant, such as a 2 mm Hg fall in blood pressure or a 0.4-point pain change when the minimal clinically important difference (MCID) is 1.5 points. Judge the effect size and its confidence interval against what matters to patients; conversely, a small trial with a nonsignificant result may simply be underpowered.

If even the favorable end of the confidence interval falls short of the MCID, the result is statistically significant but clinically unimportant.