Clinical Trial Design & Critical Appraisal

Designing out bias: randomization, concealment, and blinding

Randomization. Random allocation is what lets a randomized controlled trial support causation: with adequate numbers it spreads both measured and unmeasured prognostic factors evenly between arms. Block randomization keeps group sizes equal as enrollment proceeds, and stratified randomization balances key factors such as study site or disease stage; the baseline-characteristics table (Table 1) shows whether balance was achieved. Cluster trials randomize whole practices or hospitals to prevent contamination between arms and must be analyzed with the clustering taken into account.

Allocation concealment. Concealment hides the upcoming assignment from whoever enrolls patients (central web or telephone randomization, pharmacy control, sequentially numbered opaque sealed envelopes). If recruiters can foresee the next assignment, they can steer sicker or healthier patients into one arm, producing selection bias. Unlike blinding, concealment is always achievable, even in surgical trials.

Blinding. Masking participants and clinicians prevents unequal co-interventions and care (performance bias); masking outcome assessors prevents skewed measurement (detection bias). Blinding matters most for subjective outcomes such as pain scores and least for all-cause mortality. Identical placebos and the double-dummy technique (each arm receives one active and one placebo form when routes differ) make it possible. When participants cannot be blinded, as in surgery-versus-drug trials, use blinded outcome adjudicators and hard endpoints.

Who is analyzed: intention-to-treat, per-protocol, and attrition

Intention-to-treat (ITT). Analyzing everyone as randomized, regardless of adherence, crossover, or withdrawal, preserves the balance randomization created. Because non-adherence makes the arms look more alike, ITT biases toward the null, which makes it conservative and the standard primary analysis for superiority trials.

Per-protocol and as-treated. Adherence is itself a marker of health: patients who faithfully take even a placebo have lower mortality (the healthy-adherer effect). Restricting analysis to adherers therefore reintroduces confounding and usually inflates benefit. In a noninferiority trial, however, ITT's drift toward "no difference" favors the new treatment, so ITT and per-protocol results should both be reported and should agree.

Attrition bias. Loss to follow-up matters most when it is unequal between arms or tied to the outcome, such as side effects driving dropouts in one arm. By a common rule of thumb, under 5% loss causes little bias and over 20% seriously threatens validity. Sensitivity analyses such as worst-case imputation test whether conclusions survive, and the CONSORT flow diagram should account for every randomized patient.

Superiority, noninferiority, and equivalence

Superiority. The trial asks whether the new treatment is better; the result is significant when the 95% CI excludes the no-effect value (1 for ratios, 0 for differences). A nonsignificant superiority trial shows only that no difference was detected, often because it was underpowered; it does not prove equivalence.

Noninferiority. The trial asks whether a treatment with other advantages (oral route, fewer adverse effects, lower cost) is not worse than a proven active control by more than a margin. The margin must be prespecified and clinically justified, usually preserving a fixed fraction of the comparator's established benefit over placebo, and the comparator must be given at its proven dose; a weak or underdosed comparator makes almost anything look noninferior. For a ratio where values below 1 favor the new drug (for example, an HR margin of 1.3):

Once noninferiority is shown, testing superiority with the same CI is acceptable; claiming noninferiority after a failed superiority trial requires a margin chosen in advance.

Equivalence. The entire CI must fall between margins on both sides of no effect. Generic drugs demonstrate bioequivalence when the 90% CI for the generic-to-brand ratio of AUC and Cmax lies within 80–125%.

Endpoints, multiplicity, and early stopping

Primary and secondary endpoints. The prespecified primary endpoint drives the sample-size calculation and carries the conclusion. Secondary endpoints are supportive, and when the primary endpoint fails they are only hypothesis-generating. Registering outcomes before enrollment (for example, on ClinicalTrials.gov) exposes outcome switching and selective reporting.

Composite endpoints. Combining outcomes such as cardiovascular death, MI, stroke, and urgent revascularization raises event rates and power, but the result can be driven by the most frequent, least important ("softest") component, such as hospitalization or revascularization, while death is unchanged. Inspect each component.

Surrogate endpoints. HbA1c, LDL, blood pressure, tumor response or progression-free survival, viral load, and bone density are faster and cheaper to measure than patient-important outcomes (death, MI, stroke, fracture, quality of life), but improving a marker does not guarantee benefit. In CAST, encainide and flecainide suppressed ventricular ectopy after MI yet increased mortality; in ACCORD, intensive HbA1c lowering increased deaths.

Multiple comparisons. Every additional test adds false-positive risk. Protections include prespecified hypotheses, hierarchical (gatekeeping) testing in which secondary endpoints are tested only after the primary succeeds, and the conservative Bonferroni correction.

Subgroup analyses. Post hoc subgroup effects (by sex, age, or region) are often chance findings, especially when many subgroups are examined. A credible subgroup effect is prespecified with a predicted direction, supported by a significant test for interaction (not merely P < 0.05 in one subgroup and not the other), biologically plausible, and replicated. ISIS-2 made the point deliberately: aspirin appeared not to help patients born under Gemini or Libra. When the interaction test is nonsignificant, the overall result is the best estimate for each subgroup.

Data and safety monitoring boards. An independent DSMB reviews unblinded interim data and recommends stopping for clear benefit, harm, or futility, using boundaries that are stringent early (O'Brien-Fleming, or Haybittle-Peto P < 0.001) to preserve the overall α.

Trials stopped early for benefit, especially after few events, tend to overestimate the treatment effect because they stop on a random high.

Reading Kaplan-Meier curves, hazard ratios, and forest plots

Kaplan-Meier curves. The y-axis shows the proportion still event-free over time. Each downward step is an event; each tick mark is a censored patient who left observation without an event (lost, withdrawn, or event-free at study end). Median survival is read where a curve crosses 0.5, and the log-rank test compares whole curves. Check the number-at-risk table: the right tail rests on few patients and is unreliable.

Hazard ratios. A Cox proportional-hazards model yields the ratio of instantaneous event rates in the two arms; an HR of 0.75 means a 25% lower hazard throughout follow-up. The model assumes the ratio is constant, so curves that cross violate the assumption.

Forest plots. Each square is a study's or subgroup's point estimate, sized by its weight, and the horizontal line through it is the 95% CI. The vertical line of no effect sits at 1 for ratios and 0 for differences; the diamond is the pooled estimate, its width spanning the pooled CI.

For any ratio (RR, OR, HR), a 95% CI that includes 1 means the result is not statistically significant (P ≥ 0.05); for a difference, the corresponding value is 0.

Meta-analysis and applying results to your patient

Heterogeneity. I² estimates the percentage of between-study variation not due to chance (roughly 25%, 50%, and 75% indicate low, moderate, and high heterogeneity); Cochran's Q test has low power when studies are few. Substantial heterogeneity calls for a random-effects model, a search for sources such as population, dose, or design, or a decision not to pool.

Publication bias. A funnel plot graphs each study's effect against its precision or size. Without bias, large precise studies cluster near the pooled estimate at the top and small studies scatter symmetrically at the base. A missing corner of small null or harmful studies suggests publication bias, which inflates the pooled effect; Egger's test quantifies the asymmetry, and searching trial registries recovers unpublished results. Asymmetry can also reflect small-study effects, since smaller trials may enroll different patients or be of lower quality.

External validity. Internal validity asks whether the result is true for the patients studied; external validity asks whether it applies to your patient. Trials often exclude older adults and patients with CKD, pregnancy, or multimorbidity, and single-center academic settings and placebo run-in periods that remove nonadherent or intolerant patients make results look better than they will in practice. An excluded patient is outside the evidence, however well the trial was run.