Why anyone cares: Of 53 landmark publications in cancer, 47 could not be replicated)
Heck, we’ve only got started with the notion of trying to replicate (reproduce) “edgy” findings, and already we’re hearing about all the damage it might do.
Good call for protecting backsides. Who’s had anything like time to assess all the damage from taking the edgy findings seriously? And now we are already being warned to go slow on reform. From an op-ed in Nature:
So why am I concerned? Isn’t reproducibility the bedrock of the scientific process? Yes, up to a point. But it is sometimes much easier not to replicate than to replicate studies, because the techniques and reagents are sophisticated, time-consuming and difficult to master. In the past ten years, every paper published on which I have been senior author has taken between four and six years to complete, and at times much longer. People in my lab often need months — if not a year — to replicate some of the experiments we have done on the roles of the microenvironment and extracellular matrix in cancer, and that includes consulting with other lab members, as well as the original authors.
People trying to repeat others’ research often do not have the time, funding or resources to gain the same expertise with the experimental protocol as the original authors, who were perhaps operating under a multi-year federal grant and aiming for a high-profile publication. If a researcher spends six months, say, trying to replicate such work and reports that it is irreproducible, that can deter other scientists from pursuing a promising line of research, jeopardize the original scientists’ chances of obtaining funding to continue it themselves, and potentially damage their reputations.
Hey, wait a minute. What about the risk of sacrificing one’s lifespan, comfort, and estate on cancer treatments that don’t work and could have been shown not to work if replication studies were more frequently undertaken?
If it is really true that colleagues in a highly specialized area can’t produce the same results straightforwardly, why should we expect oncologists and nurse practitioners serving the general public to do so? It’s true, we mustn’t jump to conclusions about the researchers just because a few results can’t be replicated. Here, as everywhere, it is the pattern that counts. Which is why replication is important.
See also: The economist weighs in on broken peer review
What never existed can’t be replicated
Hat tip: Bioethics.com