By Steven Milloy
November 19, 2004
“Increases in air pollution caused by cars, power plants and industry can be directly linked to higher death rates in U.S. cities, a study said,” reported Reuters this week.
The Reuters reporter, I suppose, had no hope of taking the study’s results to task (as they beg to be) since she was undoubtedly hypnotized by the ostensible prestige of the journal in which the study was published— the Nov. 17 issue of the Journal of the american Medical association— the seemingly mesmerizing affiliations of the study’s authors (Yale University and the Johns Hopkins University), and the sleep-inducing nature of the study’s statistical analysis.
But had the reporter been able to go beyond simple regurgitation of the study’s press release, Reuters’ might well have reported “Researchers tried to scare public with statistical malpractice.”
The researchers compared the non-injury-related death rates and smog measurements for 95 urban areas for the period 1987-2000. They reported a one-half percent (0.5 percent) increase in premature death (mortality) per 10-part per billion increase in ground-level ozone (smog) in the urban areas. Reducing smog levels by 35 percent, they claim, could save about 4,000 lives per year.
While no one likes smog and all of us would like to see smog levels reduced where possible, we shouldn’t let that fact amount to critical immunity for flawed research or for “anything goes” conclusions like these.
First, if smog is deadly in New York City, then it should be deadly everywhere. But even granting the researchers every benefit of the doubt with respect to the validity of their analysis, among the 95 urban areas included in the study, the correlation between smog and mortality is only statistically meaningful in five of those 95 urban areas (New York City, Newark, Philadelphia, Dallas-Ft. Worth and Chicago).
That means in 95 percent of the urban areas studied, there was no meaningful correlation between smog and mortality.
It’s simply not credible that smog would be a killer in five particular cities, but nowhere else.
But, of course, I’m not going to give the researchers the benefit of the doubt with their analysis. It simply stinks.
The type of study undertaken — called an “ecologic” type study by epidemiologists — is fundamentally incapable of linking smog with mortality.
Not a single death was specifically linked by the researchers to smog. In no case was there a medical finding that anyone’s death was, in fact, caused by smog. The researchers have no idea how much smog to which any of the people in the study were exposed.
Rather, the researchers only compared, on a very macro level, urban death rates and urban smog levels. They did not look to see whether individuals exposed to higher levels of smog had greater rates of premature death after ruling out all other likely risk factors for premature death.
It is taught in Epidemiology 101 that ecologic studies are very crude tools that, at best, may be used to develop ideas for further research. The smog study’s authors know this, too.
Jonathan Samet, one of the authors, once discouraged the use of ecologic studies, writing in the journal Health Physics that, “The methodologic limitations inherent in the ecologic method may substantially bias ecologic estimates of risk.”
The study’s reported increase in risk of 0.52 percent per 10 ppb of smog is laughably small — so small that it probably could not be reliably identified by the researchers.
as the National Cancer Institute once put it, “In epidemiologic research, [increases in risk of less than 100 percent] are considered small and usually difficult to interpret. Such increases may be due to chance, statistical bias or effects of confounding factors that are sometimes not evident.” The smog study’s reported increase in risk is less than 1 percent — 100 times less than a minimally reliable level.
This study, in reality, reported no association between smog levels typically found in U.S. urban areas and mortality.
What’s really going here is yet another example of a U.S. Environmental Protection agency-funded ongoing effort to churn out one junk science-fueled alarm after another regarding air quality. The purpose is to grease the skids for the EPa to issue more stringent air quality regulations in the future — standards that provide the agency with more power over states. Federal funding of state highway projects, after all, is tied to whether states meet the EPa’s air quality standards.
Oh, and did I mention that the EPa’s standards tend to be costly — the agency’s last round of rules issued in 1997 is estimated to cost $100 billion per year when fully implemented — and produce few, if any, measurable public health benefits?
There is no argument that smog can be very unpleasant, and in some situations, smog may have adverse health effects on some vulnerable parts of city populations — in EPa-speak, these people are flippantly referred to as “elderly, urban, and asthmatic joggers.” That said, there isn’t any evidence that the levels of smog experienced in the U.S. today are killing anyone — once you clear up the smoggy statistics, that is.