Retractions of scientific papers are on a steep rise. They join a string of science scandals ranging from Andrew Wakefield's study linking childhood vaccine and autism to the allegations that Marc Hauser, once a star psychology professor at Harvard, fabricated data for research on animal cognition, reports Bourree Lam. (1)
By one estimate, from 2001 to 2010, the annual rate of retractions by academic journals increased by a factor of 11 after adjusting for increases in published literature, and excluding articles by repeat offenders. (2)
Dr. John Ioannidis, a director of Stanford University's Meta-Research Innovation Center, who once estimated about half of published results across medicine were inflated or wrong, noted the proportion in psychology was even larger than he had thought. He said the problem could be even worse in other fields, including cell biology, economics, neuroscience, clinical medicine, and animal research. A years long effort to reproduce 100 studies published in three leading psychology journals has found that more than half of the findings did not hold up when retested. (3)
A study in the
Proceedings of the National Academy of Science reviewed 2,047 retractions of biomedical and life-sciences articles and found that just 21.3 percent stemmed from straightforward error, while 67.4 percent resulted from misconduct, including fraud or suspected fraud and plagiarism. (4)
An important point! Being contradicted by follow-up studies and meta-analyses doesn't prevent a paper from being used as though it were true. Even effects that have been contradicted by massive follow-up trials with unequivocal results are frequently cited 5 to 10 years later, with scientists apparently not noticing that the results are false. (5) Of course, new findings get widely publicized in the press, while contradictions and corrections are hardly ever mentioned. (6)
Why?
This surge raises an obvious question: Are retractions increasing because errors and other misdeeds are becoming more common, or because research is now scrutinized more closely? (3)
In a number of cases, the explanation for the discrepancies may lie in precisely what you'd suspect, sample size. The smaller the group, the shorter the study, the more likely it may be that subsequent, deeper, investigation will contradict or alter the original thesis.
Also, consider that top-ranked journals, such as
Nature and Science, prefer to publish studies with ground-breaking results—meaning large effect sizes in novel fields with little prior research. This is a perfect combination for chronic truth inflation. Some evidence suggests a correlation between a journal's impact factor (a rough measure of its prominence and importance) and the factor by which its studies overestimate effect sizes. Studies that produce less 'exciting' results are closer to the truth but less interesting to a major journal editor. (7, 8)
Many scientists blame increased competition for academic jobs and research funding, combined with a 'publish or perish' culture. Because journals are more likely to accept studies reporting 'positive' results (those that support, rather than refute, a hypothesis), researchers may have an incentive to 'cook' or 'mine' their data to generate a positive finding. As long ago as 1987, a study found that, compared with research trials that went unpublished, those that were published were three times as likely to have positive results. (9)
Mara Burney sums this up well. “All of this does not mean that medical studies are of no value or that health reports are always wrong. It simply serves as a caution that science is fluid, not static or absolute. A little skepticism may be just what the doctor ordered.” (10)
Lastly, even in earlier times folks had problems like these as evidenced from these two quotes:
- “If your doctor does not think it is good for you to sleep, drink wine, or eat of a particular dish, do not worry; I will find you another who will not agree with him.” Michel Eyquem de Montaigne (1533-92)
- “Part of the secret of success in life is to eat what you like and let the food fight it out inside.” Mark Twain
Jack Dini
Livermore, CA
References
1. Bourree Lam, “The new science of bad science,”
The Atlantic, September 2015
2. Michael L. Grieneisen and Minghua Zhang, “A comprehensive survey of retracted articles from the scholarly literature,”
PloS One, October 2012
3. Benedict Carey, “Psychology's fears confirmed: rechecked studies don't hold up,”
The New York Times, August 28, 2015
4. Ferric C. Fang et al., “Misconduct accounts for the majority of retracted scientific publications,”
Proceedings of the National Academy of Science, October 2012
5. A. Tatsioni et al., “Persistence of contradicted claims in the literature,”
JAMA, No. 21, 2517, 2007
6. F. Conon et al., “Why most biomedical findings echoed by newspapers turn out to be false: The case of attention deficit hyperactivity disorder,”
PloS One, 7, No. 9, 2012, e44275
7. B. Brembs et al., “Deep impact: unintended consequences of journal rank,”
Frontiers in Human Neuroscience, 7, 291, 2013
8. K,. C. Siontis et al., “Magnitude of effects in clinical trials published in high-impact general medical journals,”
International Journal of Epidemiology, 40, No.5, 1280, 2011
9. K. Dickersin et al., “Publication bias and clinical trials,”
Controlled Clinical Trials, December 1987
10. Mara Burney, “Don't believe everything you read—even in medical journals,'' healthfacts
>andfears.com, July 15, 2005
