Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time.
Level 4 - case-series / case-control
Level 4 by design analogy: meta-epidemiological retrospective cohort study of published clinical trials.
PubMed 26244868 · doi:10.1371/journal.pone.0132382
What was done
Authors identified all large NHLBI-funded randomized controlled trials published between 1970 and 2012 evaluating drugs or dietary supplements for adult cardiovascular disease prevention or treatment, with direct costs exceeding $500,000 per year and primary outcomes of cardiovascular risk, disease, or death. The 55 qualifying trials were evaluated for publication date (before vs. after 2000), pre-registration in ClinicalTrials.gov, comparator type (active vs. placebo), industry co-sponsorship, and whether primary outcomes and total mortality were positive, negative, or null.
What was found
Prior to 2000, 17 of 30 trials (57%) reported a significant benefit on the primary outcome, compared to only 2 of 25 trials (8%) published after 2000 (χ² = 12.2, df = 1, p = 0.0005). The proportion of trials utilizing active versus placebo comparators remained constant over time, and industry co-sponsorship was not related to reporting significant benefit. Pre-registration in ClinicalTrials.gov was strongly associated with the shift toward null findings.
Why it matters
This study provides empirical evidence that prospective trial registration and transparent outcome specification are associated with a dramatic reduction in positive findings, likely by curbing selective outcome reporting and publication bias.
Limits
The analysis is restricted to 55 large, NHLBI-funded cardiovascular drug and supplement trials, which may not generalize to smaller trials, other therapeutic areas, non-NIH funding mechanisms, or non-pharmacological interventions. The observational before-and-after design cannot definitively prove that trial registration caused the reduction in positive outcomes rather than concurrent shifts in drug discovery pipelines or trial methodologies.
Cited by
- supports Prior to 2000, most large trials for cardiovascular medications and supplements were positive, but after pre-registration of predictions was mandated in 2000, almost all subsequent trials yielded negative results.