For decades, researchers have complained that one of the problems that compromises research today is null findings – that is, if you don’t prove your point, you’re a failure, so you bury it (the file drawer problem). But that attitude shows poor collective reasoning skills. Showing which answers are wrong is, logically, a step on the road to right answers.
A proposed reform has been registering studies and peer reviewing the protocols, so that whatever the researchers find will count as a finding. They don’t have to find something when there’s nothing, as long as the hunt is agreed to be worthwhile in principle, justifiable. What happened when it was tried?
To see whether registered reports increase the frequency at which null results are reported, psychologists Chris Allen and David Mehler analysed the outcomes of 113 registered reports in the biomedical and psychological sciences.
The pair identified 296 discrete hypotheses across those studies, and found that, overall, 61% of these were not supported by the results that those studies later published. For studies that sought to replicate previous findings, the percentage of null results was slightly higher, at 66%, whereas this figure stood at 55% for original research (see ‘Registered reports cut publication bias’).Matthew Warren, “First analysis of ‘pre-registered’ studies shows sharp rise in null findings” at Nature
It became easier to admit that one’s hypothesis “needs work” or however you want to put it.
It’s great to see somebody actually trying to do something about the long-standing file drawer problem. In general, the pace of reform in science publication seems, at times, to match the pace we see at the Vatican. But science publishing doesn’t have the excuse of antiquity…
Follow UD News at Twitter!
See also: What can a huge retractions database teach us? Overall, improved vigilance has slowed the trend, but key problems remain, including manipulated images. If a picture is worth a thousand words, that’s about three to five paragraphs of falsehood.