Precision, Reliability & Combined Testing
Precise is not the same as right, and pairing tests trades sensitivity for specificity
Accuracy versus precision
Accuracy (validity). Accuracy is how close a measurement, or a study's estimate, comes to the true value. It is eroded by systematic error: bias that pushes every result in the same direction, such as a mis-zeroed scale or a blood-pressure cuff too small for the arm (falsely high readings).
Precision (reliability, reproducibility). Precision is how closely repeated measurements of the same quantity agree. It is eroded by random error, the unpredictable scatter above and below the mean from sampling variation, instrument noise, or observer inconsistency. Precision alone says nothing about whether the average is correct.
The bullseye. A tight group on the center is accurate and precise. A tight group away from the center is precise but inaccurate, the signature of a miscalibrated instrument that errs the same way every time. A loose spray centered on the bullseye is accurate on average but imprecise, and a loose spray off to one side is neither.
| Feature | Accuracy (validity) | Precision (reliability) |
|---|---|---|
| Question asked | Is the result correct? | Is the result reproducible? |
| Degraded by | Systematic error (bias), in one direction | Random error, in both directions |
| Improved by | Calibration against a reference standard, better design, blinding | Larger samples, repeated or averaged measurements, standardized technique |
Reliability is necessary but not sufficient for validity: a precise instrument can be consistently wrong, but one that cannot reproduce its own results cannot be accurate.
Random error, sample size, and bias
What more data can fix. Random error shrinks as observations are pooled. The standard error of a mean equals SD/√n, so confidence-interval width narrows with the square root of the sample size: quadrupling enrollment roughly halves the interval. Averaging repeat measurements in one patient, such as two or three blood-pressure readings, improves precision the same way.
What more data cannot fix. Systematic error does not average out. A very large study with selection bias or a miscalibrated instrument yields a narrow confidence interval centered on the wrong value: precisely wrong. Bias is prevented by design, through random allocation, blinding, standardized protocols, and calibration against a gold standard.
Measuring reliability
| Type | Question | Example | Usual statistic |
|---|---|---|---|
| Test–retest | Same result over time in stable subjects? | Questionnaire repeated two weeks apart | Correlation or intraclass correlation |
| Intrarater | Does one observer agree with themselves? | Radiologist rereads the same films, blinded to the first read | κ |
| Interrater | Do different observers agree? | Two pathologists grade the same slides | Cohen κ (categorical); intraclass correlation (continuous) |
| Internal consistency | Do an instrument's items measure one construct? | Items on a depression rating scale | Cronbach α |
Cohen κ. Raw percent agreement overstates reliability because observers agree some of the time by luck: two readers who each call 90% of films normal agree on about 82% of films (0.9 × 0.9 + 0.1 × 0.1) even when guessing. Kappa removes this chance component: κ = (observed agreement − expected agreement)/(1 − expected agreement). A κ of 0 means agreement no better than chance, 1 means perfect agreement, and negative values mean worse than chance. By the widely used Landis–Koch benchmarks, 0.41–0.60 is moderate, 0.61–0.80 substantial, and above 0.80 near-perfect.
Cronbach α. Internal consistency applies to multi-item instruments such as symptom questionnaires. Cronbach α ranges from 0 to 1; about 0.7 or higher is conventionally acceptable, and very high values can signal redundant items.
Combining diagnostic tests
Parallel testing. Both tests are done at once, and the patient is called positive if either is positive. Disease gets two chances to be detected, so sensitivity and NPV rise, while specificity (and usually PPV) falls because false positives from either test count. Parallel testing suits settings where missing a serious disease is unacceptable or an answer is needed quickly, such as emergency and inpatient evaluations; a negative result on both tests helps rule the disease out.
Serial testing. The second test is done only when the first is positive, and the patient is called positive only if both are positive. False positives from the first test are usually caught by the second, so specificity and PPV rise, while sensitivity (and NPV) fall because a case missed by either test is lost. Serial testing is economical, reserving the costlier or more invasive test for the few who screen positive, and suits nonurgent outpatient screening where a false-positive label causes harm. The cheap, highly sensitive test goes first; the highly specific confirmatory test comes second.
| Feature | Parallel (either positive) | Serial (both positive) |
|---|---|---|
| Sequence | Tests run simultaneously | Second test only after a positive first test |
| Increases | Sensitivity, NPV | Specificity, PPV |
| Decreases | Specificity, PPV | Sensitivity, NPV |
| Net sensitivity | 1 − (1 − Se1)(1 − Se2) | Se1 × Se2 |
| Net specificity | Sp1 × Sp2 | 1 − (1 − Sp1)(1 − Sp2) |
| Best use | Ruling out a dangerous disease quickly | Confirming a diagnosis at lower cost |
Worked example. Test A has sensitivity 80% and specificity 90%; test B has sensitivity 90% and specificity 95%. In parallel, net sensitivity is 1 − (0.2 × 0.1) = 98% and net specificity is 0.90 × 0.95 = 85.5%. In series, net sensitivity is 0.8 × 0.9 = 72% and net specificity is 1 − (0.1 × 0.05) = 99.5%.
Two tests that share a mechanism, such as two similar antibody assays, tend to miss the same patients, so combining them gains less than the independence arithmetic predicts.
Serial testing in practice
HIV. The current CDC laboratory algorithm is a serial design:
- HIV-1/2 antigen/antibody immunoassayfourth-generation screen detecting p24 antigen plus antibodies; highly sensitive, with a shorter window period
- HIV-1/HIV-2 antibody differentiation immunoassayrun only on reactive screens; confirms infection and distinguishes HIV-1 from HIV-2
- HIV-1 nucleic acid test (RNA)run when the differentiation assay is negative or indeterminate; detects acute infection, when p24 antigen precedes antibodies
The Western blot, once the confirmatory step, is no longer used.
Syphilis. The traditional sequence screens with a nontreponemal test (RPR or VDRL), which is cheap and sensitive but can be falsely positive in pregnancy, lupus or antiphospholipid syndrome, and viral infections, then confirms reactive results with a treponemal test (FTA-ABS or TP-PA). The reverse-sequence algorithm starts with an automated treponemal immunoassay, follows reactive results with an RPR, and resolves discordance with TP-PA.
Breast cancer. Screening mammography is the sensitive first step; biopsy, specific but invasive, is reserved for suspicious findings (BI-RADS 4 or 5). Abnormal cervical cytology followed by colposcopy follows the same screen-then-confirm logic.