Most clinical trials conducted by researchers at US academic medical centers did not get reported or published results within two years of completion.
A large study published in the British Medical Journal reviewed thousands of preliminary studies and found an astoundingly large fraction of them never came to publication light, leading to major distortions in public health policy and medical practice. (1)
The researchers reviewed all interventional clinical trials with a primary completion date between October 2007 and September 2010, and with a lead investigator affiliated with an academic medical center. Their goal was to elucidate the publication or reporting of results within 24 months of study completion. They reviewed over 4,300 interventional trials based at 51 US trial experienced academic medical centers. The surprising findings were than only 36 percent reported or published results within two years of study completion, and fully one-third did not report or publish results at all in the following years.
The impact of this lack of data reporting means only a biased slice of research of information has been influencing medicine and future research. The authors concluded that “despite the ethical mandate and expressed values and mission of academic institutions, there is poor performance and noticeable variation in the dissemination of clinical trial results across leading academic medical centers.” (2)
The reason why the public and academic institution and public health policymakers should be alarmed about these results is that when null/negative findings are ignored or trashed rather than being published, the medical literature is skewed toward those perhaps relatively few studies that do show an effect of a treatment or behavior.
This is known among scientists as 'publication bias.' If 90 percent of studies show no effect and 10 percent show some effect, but the 90 percent never see the light of day, obviously a false picture of benefit or efficacy will be the bottom line. This will redound to the detriment of public health: the mandate to publish all human trial data should be rigorously enforced in the name of science and sound public policy reports Gil Ross. (2)
As long ago as 1987 a study found that, compared with research trials that went unpublished, those that were published were three times as likely to have positive results. (3)
Many scientists blame increased competition for academic jobs and research funding, combined with a 'publish or perish' culture. Because journals are more likely to accept studies reporting 'positive' results (those that support rather than refute a hypothesis), researchers may have an incentive to 'cook' or 'mine' their data to generate a positive finding.
Couple this with the information that many published results cannot be duplicated leads one to wonder about the state of published medical research.
For example, in 2015 John Ioannidis reported that one-third of studies published in three reputable peer reviewed journals didn't hold up. He looked at 45 studies published between 1990 and 2003 and found that subsequent research contradicted the results of seven of those studies, and another seven were found to have weaker results than originally published. In other words, 32% did not withstand the test of time. (4)
Since then researchers in several scientific areas have consistently struggled to reproduce major results of prominent studies. By some estimates, at least 51%, and as much as 98% of published papers are based on studies and experiments showing results that cannot be reproduced. (5)
Support Canada Free Press
The Institute of Medicine estimates that only 4 percent of treatments and tests are backed up by strong scientific evidence; more than half have very weak evidence or none. (6)
Psychology has become something of a poster child for the reproducibility crisis. Brian Nosek, a psychology professor at the University of Virginia, coordinated a Reproducibility Initiative project to repeat 100 psychological experiments and could only successfully replicate 40% (5)
Pharmaceutical companies find that test results favoring new drugs typically disappear when the tests are repeated. Bayer found that two-thirds of such findings couldn't be reproduced and Amgen scientists, following up on 53 studies that at first glance looked worth pursuing, could confirm only six of them. (7)
Meanwhile, in cases where researchers have access to large amounts of data, there's a dangerous tendency to hunt for significant correlations. Researchers can thus convince themselves that they've spotted a meaningful connection, when in fact, such connections are totally random. (5)
An important point! Being contradicted by follow-up studies and meta-analyses doesn't prevent a paper from being used at though it were true.
Even effects that have been contradicted by massive follow-up trials with unequivocal results are frequently cited 5 to 10 years later, with scientists apparently not noticing that the results are false. (8)
References
Ruijun Chen et al., “Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers,” BMJ, 352, i637, February 17, 2016
Gil Ross, “Clinical trials go unreported all too often,” American Council on Science and Health, acsh.org, February, 18, 2016
K. Dickersin et al., “Publication bias and clinical trials,” Controlled Clinical Trials, December 1987
John P. Ioannidis, “Contradicted and initially stronger effects in highly cited clinical research,” JAMA, 292(2), 218, July 2005
Olivia Goldhill, “Many scientific 'truths' are, in fact, false,” qz.com, March 13, 2016
Shannon Brownlee, Overtreated, (New York, Bloomsbury, 2007), 92
Jack Dini,, “Questioning science research,” Canada Free Press, December 27, 2013
A Tatsioni et al., “Persistence of contradicted claims in the literature,” JAMA, No. 21, 2517, 2007
Jack Dini is author of Challenging Environmental Mythology. He has also written for American Council on Science and Health, Environment & Climate News, and Hawaii Reporter.
