On the low reproducibility of cancer studies
Level 5 - mechanism / opinion, no new human data
Narrative critique and commentary analyzing published replication reports; Level 5 by design analogy.
OpenAlex W2792030746 · doi:10.1093/nsr/nwy021
What was done
The authors reviewed published literature on cancer research reproducibility and analyzed the first batch of replication results from the Reproducibility Project: Cancer Biology (RP: CB). They re-evaluated the findings of 5 published replication studies, compared them with historical reproducibility estimates (such as an earlier report confirming 6 of 53 studies), and assessed whether failures to replicate stem from human technical error or intrinsic biological heterogeneity during tumorigenesis.
What was found
Historical literature cited an approximate 10% confirmation rate (6 of 53 findings) in preclinical cancer biology. In the first batch of 5 RP: CB replication studies—which the original eLife editorial classified as 2 reproducible in important parts, 1 not reproducible, and 2 uninterpretable due to control failures—the authors reclassified 4 as not reproducible and 1 as uninterpretable because control failures reflect failure to reproduce the baseline biological phenomenon. Specific examples highlighted massive baseline survival and tumor mass variability (e.g., up to 5-fold variations under identical conditions and control mice dying in 1 week versus 9 weeks originally).
Why it matters
This commentary highlights that irreproducibility in preclinical oncology may not solely reflect sloppy methodology or fraud, but could be an intrinsic biological feature of evolutionary divergence and heterogeneity across tumor models, which poses direct challenges for translating preclinical cancer findings to clinical practice.
Limits
This is a conceptual critique and narrative review of a very small sample (5 replication studies), not a systematic meta-analysis or primary experimental study. Detailed quantitative metrics for the full RP: CB cohort (which targeted 30 studies) were incomplete at the time of this publication.
Cited by
- context Approximately 40% of scientific publications cannot be replicated.