Should we believe them when they tell us that the drinking four cups of coffee daily lowers our risk of death people with spouses live longer? A statistician and a physician team up to
explain why not:
A subtler manifestation of dishonesty in research is what amounts to statistical cheating. Here is how it works… If you try to answer one question – by asking about levels of coffee consumption, for example, to test whether drinking certain amounts a day are associated with more or less cancer; or whether being married is associated with increased longevity — and test the results with appropriate statistical methods, there is a 5% chance of getting a (nominally) statistically significant result purely by chance (meaning that the finding isn’t real).
If you try to answer two questions, the probability is about 10%. Three questions, about 14%. The more you test, the more likely you’ll get a statistical false positive, and researchers exploit this phenomenon. Researchers have thereby created what is for them a winning “science-business model”: Ask a lot of questions, look for associations that may or may not be real, and publish the result. Just how many questions are typical in a university research project? It varies by subject area, and researchers have become masters at gaming their methodology. Often, they ask thousands of questions. And they get away with it, because there are no scientific research cops … S. Stanley Young & Henry Miller, “Junk Science Has Become a Profitable Industry. Who Will Stop It?” at RealClearScience
A writer raises another point:
Science “journalists” are a huge part of the problem. If the talking points make for a good headline, like Young and Miller’s example about coffee, then the “research” will get written up, no questions asked.
Young and Miller ultimately ask what can be done to save science, and suggest the solution might fall on government funding agencies “to cut off support for studies with flawed design; and to universities, which must stop rewarding the publication of bad research.” But since so many studies these days just confirm preconceived notions, it seems unlikely that partisans at federal agencies or congressional appropriators will stop giving funds. Ashe Schow, “‘Junk Science’ Is Everywhere, And The Media Eat It Up” at Daily Wire
Not only that, one might add that many media feel a sense of virtue about running with this stuff. Unlike hearing the latest and greatest on a celeb bust-up, it’s supposed to be good for people’s health. But of course, the reality is that if it’s just misinformation, it could be worse for people’s health than hearing the latest about the bust-up.
Follow UD News at Twitter!
See also: From RealClearScience: No, we can’t trust government data on diet and nutrition. Censored researchers: Nutrition is a “degenerating” research paradigm. Also: The skinny on salt, veggie oil, skim milk, whole foods. Nutrition science is nearly baseless but it rules.