Remember the Sokal Hoax? A physics professor manages to sneak in a completely garbage paper to a “postmodern cultural studies” journal? Well, if you thought that science journals were immune to this sort of thing – or even more often than not reliable – then get ready to have some of your faith in the modern academia broken up a bit.
Who’s Afraid of Peer Review? documents the story, and here’s a bit of the opening:
On 4 July, good news arrived in the inbox of Ocorrafoo Cobange, a biologist at the Wassee Institute of Medicine in Asmara. It was the official letter of acceptance for a paper he had submitted 2 months earlier to the Journal of Natural Pharmaceuticals, describing the anticancer properties of a chemical that Cobange had extracted from a lichen.
In fact, it should have been promptly rejected. Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s short-comings immediately. Its experiments are so hopelessly flawed that the results are meaningless.
Sounds like some pretty obvious garbage. But hey, just how many journals could have possibly let this one slip by the gates?
By the time Science went to press, 157 of the journals had accepted the paper and 98 had rejected it. Of the remaining 49 journals, 29 seem to be derelict: websites abandoned by their creators. Editors from the other 20 had e-mailed the fictitious corresponding authors stating that the paper was still under review; those, too, are excluded from this analysis. Acceptance took 40 days on average, compared to 24 days to elicit a rejection.
Now, it was open access journals that were targeted here. But the results aren’t all that encouraging for ‘traditional’ journals either. With these results in mind, here’s a question worth considering. If an obviously nonsense paper really can make it into so many journals, what are the odds that papers with more subtle but still serious flaws are slipping past the gates?