Peer review

Robert J. Marks, author with design theorist William Dembski and Winston Ewert of Introduction to Evolutionary Informatics talks with Gary Smith, author The AI Delusion, about how, in general, based data is produced Smith: Texas Sharpshooter Fallacy # 1 is that I’m going to prove what a great shot I am and so I stand outside a barn and then I go and paint a thousand targets on the barn and I fire my gun and what do you know, I am lucky, I hit a target. And then I go and erase all the other targets and I say, look, I hit the target. And, of course, it’s meaningless, because, with so many targets, I’m bound to hit something… Read More ›

The National Academies of Science is wading into the longstanding mess over the validity of research findings. It doesn’t, of course, agree that there is a “crisis.” That said, the report also notes that the American public’s confidence in science hasn’t wavered at all in recent years, despite major news articles discussing the “crisis” in psychology and elsewhere. And it found that even scientists who have criticized the current state of things aren’t completely on-board with calling science broken. “How extensive is the lack of reproducibility in research results in science and engineering in general? The easy answer is that we don’t know,” Brain Nosek, co-founder and director of the Center for Open Science, told the report committee during a Read More ›

Researchers, says an experimental psychologist, generally know what they should do: Yet many researchers persist in working in a way almost guaranteed not to deliver meaningful results. They ride with what I refer to as the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known). My generation and the one before us have done little to rein these in.Dorothy Bishop, “Rein in the four horsemen of irreproducibility” at Nature That’s interesting, considering how often we were ordered to see science as the relentless pursuit of truth. If we start with something as basic as giving up gimmicks, maybe we’ll get further. She offers some thoughts on suggested reforms. Follow Read More ›