Uncommon Descent Serving The Intelligent Design Community

At ScienceNews: Statistical significance as a strange idea


Bruce Bower thinks that social sciences researchers wanted to seem as impressive as hard science researchers, in terms of results, so they developed the p-value (p < .05) in the mid-20th century. He doesn’t think it was a good idea.

Psychologists in particular wanted a statistical skeleton key to unlock true experimental insights. It was an unrealistic burden to place on statistics, but the longing for a mathematical seal of approval burned hot. So psychology textbook writers and publishers created one, and called it statistical significance.

By calculating just one number from their experimental results, called a P value, researchers could now deem those results “statistically significant.” That was all it took to claim — even if mistakenly — that an interesting and powerful effect had been demonstrated. The idea took off, and soon legions of researchers were reporting statistically significant results.

To make matters worse, psychology journals began to publish papers only if they reported statistically significant findings, prompting a surprisingly large number of investigators to massage their data — either by gaming the system or cheating — to get below the P value of 0.05 that granted that status. Inevitably, bogus findings and chance associations began to proliferate.

Bruce Bower, “How the strange idea of ‘statistical significance’ was born” at ScienceNews (August 12, 2021)

We know. It hasn’t helped the profession’s reputation. Some want to just dump “the null ritual”:

It’s well past time to dump the null ritual, says psychologist and applied statistician Richard Morey of Cardiff University, Wales. Researchers need to focus on developing theories of mind and behavior that lead to testable predictions. In that brave new scientific world, investigators will choose which of many statistical tools best suits their needs. “Statistics offer ways to figure out how to doubt what you’re seeing,” Morey says.

Bruce Bower, “How the strange idea of ‘statistical significance’ was born” at ScienceNews (August 12, 2021)

Bower provides an interesting account of an attempt tp use p-values to assess whether people lost their religious beliefs while contemplating Rodin’s “Thinker” statue. We can all think of more useful enterprises for social sciences than that.

Peace & joy. For those of us who used statistical analysis of facts about THINGS (e.g., failure of piece parts in an engine), the various bits of data collection and analysis were (are still?) important for effectively (yeah, right) justifying and expending budgets. The problem with trying to apply techniques that work for oil filter replacement rates to groups of people is that the people are dumber than oil filters because the people get bored and INVENT new ways to screw up. The other problem is that the oil filter is always gonna act pretty much like an oil filter, but humans get bored and INVENT new ways to "fail". That's why ya wanna AUTOMATE as much of the human's job as possible. Because the human Managers INSIST on making decisions about processes that work just on fine on "automatic". That is, the "manager" believes that his job is "Make Decision". And if the lowest level clerk can accurately make the only decision to be made, the Manager is NOT happy. mahuna

Leave a Reply