Bower: To make matters worse, psychology journals began to publish papers only if they reported statistically significant findings, prompting a surprisingly large number of investigators to massage their data — either by gaming the system or cheating — to get below the P value of 0.05 that granted that status. Inevitably, bogus findings and chance associations began to proliferate.
Smith: When I was a young assistant professor at Yale, one of my senior colleagues, Nobel Laureate James Tobin, wryly observed that the bad old days when researchers had to do calculations by hand were actually a blessing. The effort was so great that people thought hard before calculating. They put theory before data. Today, with terabytes of data and lightning-fast computers, it is too easy to calculate first, think later.
In an era where even medical journals are urged to get woke, abandoning statistical significance could mean abandoning a refuge against Correct nonsense. As Brookshire writes, “Unfortunately, there is no single alternative that everyone agrees would be better for all experiments.” But that might just be what some factions want and need.
Of course, as science embraces post-modernism, “irrefutable nonsense” could be the new standard. Along with ever more strenuous demands that we trust science.