Interpreting Abstracts & Drug Advertisements
Reading past the headline to what the trial actually showed
A Reading Routine
Question first. Abstract and advertisement items look long, but each asks one or two specific things. Read the question stem first, then scan the abstract or ad for the pieces that answer it. A fixed routine prevents the most common error: picking an answer that sounds true but goes beyond the data.
- DesignRCT, cohort, case-control, or cross-sectional; randomized, blinded?
- PopulationInclusion and exclusion criteria, setting, baseline characteristics
- Intervention vs comparatorPlacebo or active drug; equivalent dose and schedule
- Primary endpoint and follow-upHard or surrogate outcome; long enough to matter?
- Result vs claimDoes the headline match the prespecified primary result?
PICOT (population, intervention, comparator, outcome, time) is the same checklist in mnemonic form.
Design, Population, and Comparator
Design sets the ceiling. Only a randomized trial supports a causal claim; cohort, case-control, and cross-sectional studies show association, so an ad saying an observational study "proves" benefit overstates it. Note whether patients and assessors were blinded and whether results were analyzed by intention to treat (everyone counted in the arm to which they were randomized, preserving randomization) or per protocol (only those who completed treatment as assigned), which can flatter a poorly tolerated drug.
Population and generalizability. Inclusion and exclusion criteria determine to whom results apply (external validity). Trials commonly exclude older adults (for example, ≥75 years), patients with CKD or liver disease, pregnant women, and those with recent cardiovascular events. A trial can be internally valid yet uninformative for a patient who would not have qualified; applying it to her is extrapolation.
Comparator. A placebo-controlled trial establishes efficacy but cannot show that the new drug beats current therapy; that requires a head-to-head trial against an active comparator. A noninferiority trial shows only that the new drug is not worse than the standard by more than a prespecified margin. When an effective treatment exists for a serious condition, placebo alone is usually unethical, so new drugs are tested against or on top of standard care.
Check the comparator's dose. Pitting a new drug against a subtherapeutic dose of the standard (or an inferior formulation or schedule) manufactures superiority, and overdosing the comparator exaggerates its side effects.
Endpoints and Follow-Up
Primary endpoint. The prespecified primary endpoint is what the sample size was calculated for and what the α level (usually 0.05) protects. If it failed, a positive secondary endpoint, subgroup, or post-hoc analysis is hypothesis-generating, not proof. Multiple comparisons breed false positives: at α = 0.05, about 1 in 20 comparisons will look "significant" by chance alone, which is why corrections such as Bonferroni (α divided by the number of tests) exist.
Surrogate vs clinical outcomes. A surrogate is a laboratory or physiologic marker assumed to predict a clinical event; common examples are LDL, HbA1c, blood pressure, bone density, and tumor shrinkage. Surrogates permit smaller, faster trials but can mislead, so patient-oriented outcomes (death, MI, stroke, hospitalization, quality of life) outrank them.
| Surrogate | Clinical outcome it should predict | Caution |
|---|---|---|
| LDL cholesterol | MI, stroke, cardiovascular death | A new LDL-lowering drug must still show fewer events |
| HbA1c | Microvascular complications, MI, death | Glucose lowering alone does not guarantee cardiovascular benefit |
| Ventricular ectopy (PVCs) | Sudden cardiac death | CAST: encainide and flecainide suppressed PVCs after MI yet increased mortality |
| Bone mineral density | Fractures | Sodium fluoride raised bone density without preventing fractures |
| Tumor shrinkage | Overall survival | Responses do not always translate into longer life |
Composite endpoints. Many trials combine outcomes (for example, death, MI, or urgent revascularization). Check which component drives the result; a composite "win" powered by the softest component does not mean fewer deaths.
Follow-up duration. A short trial cannot show effects on MI or mortality or reveal delayed toxicity. Rare adverse effects need large exposure: by the rule of three, observing about 3n patients gives a 95% chance of seeing at least one event that occurs in 1 of n patients, so a 3,000-patient program reliably detects only reactions of about 1 in 1,000 or more. Rarer reactions surface in postmarketing (phase IV) surveillance.
Doing the Numbers
Absolute measures describe what a patient actually gains; relative measures describe proportional change and stay constant across risk levels, which is why advertisements prefer them.
| Measure | Formula | Example: MI 4% on placebo, 3% on drug |
|---|---|---|
| CER, EER | Events ÷ patients in the control or experimental arm | 0.04 and 0.03 |
| RR | EER ÷ CER | 0.75 |
| RRR | ARR ÷ CER (= 1 − RR) | 25% |
| ARR | CER − EER | 1 percentage point |
| NNT | 1 ÷ ARR, rounded up | 100 |
| ARI and NNH | ARI = EER − CER for a harm; NNH = 1 ÷ ARI | Bleeding 1% → 3%: ARI 2%, NNH 50 |
Baseline risk matters. An RRR of 25% yields an ARR of 10% (NNT 10) when baseline risk is 40%, but only 1% (NNT 100) when baseline risk is 4%. Setting NNT beside NNH frames the trade-off: treating 100 patients to prevent one MI while causing about two extra major bleeds (NNH 50) requires weighing how serious each event is.
Statistical vs clinical significance. A p-value reflects how compatible the data are with chance, not the size of the effect. In a very large trial a 1 mm Hg blood pressure difference can reach p < 0.001 and still be clinically meaningless. Judge importance from the effect size, the ARR and NNT, and whether the CI excludes the smallest difference that would matter to patients. Conversely, a small trial whose wide CI spans both no effect and a large benefit is inconclusive, not negative.
Red Flags in Drug Advertisements
| Red flag | What it looks like | What to check |
|---|---|---|
| Relative-only benefit | "Cuts risk by 50%" | Absolute rates, ARR, NNT |
| Surrogate sold as outcome | "Lowers LDL," implying fewer MIs | Was a hard endpoint measured? |
| Secondary or post-hoc headline | Subgroup "win" after a failed primary | The prespecified primary endpoint |
| Unfair comparator | Standard drug at a low dose | Equivalent dosing and schedule |
| High or unequal dropout | More withdrawals in the drug arm | CONSORT diagram, intention-to-treat results |
| Significance without importance | Tiny effect in a huge sample | Effect size, NNT |
| Buried harms | Discontinuations in small print | Adverse-event table, NNH, boxed warnings |
| Conflicts of interest | Sponsor-paid or sponsor-employed authors | Design choices, independent replication |
Attrition. Loss to follow-up under about 5% rarely matters, whereas losses above about 20%, or unequal losses between arms, seriously threaten validity because those who leave usually differ from those who stay.
Sponsorship. Industry-funded trials more often reach conclusions favorable to the sponsor. Funding and author conflicts (consulting fees, stock ownership, speakers' bureaus) call for closer scrutiny of the comparator, endpoints, and analysis, not automatic rejection of the data.
Harms and fair balance. The FDA requires prescription-drug advertising to present risks in fair balance with benefits, yet the adverse-event table, discontinuation rates, contraindications, and boxed warnings often sit in the smallest print. An ad that spotlights efficacy while burying a higher discontinuation rate misleads by omission.
Reading Tables and Choosing the Answer
Baseline table. Similar age, sex, comorbidities, and risk factors across arms suggest randomization worked. A chance imbalance in a strong prognostic factor, such as more diabetic patients in the placebo arm, can confound even a randomized comparison.
Results table. A ratio whose 95% CI includes 1, or a difference whose CI includes 0, is not significant at α = 0.05, whatever the accompanying text implies; compare each p-value with the prespecified α.
The best answer restates only what the study showed: this population, this comparator, this endpoint, this duration. Eliminate options that extend the result to excluded patients, to hard outcomes that were never measured, to other drugs in the class, or to longer use than was studied.