Intelligent Design Peer review

Big (?) Surprise: Cool, glitzy papers less likely to be replicated

Spread the love

Well, it’s good that some are seeing it as a crisis and not just as the way business is done:

Summary: Researchers have uncovered another problem with a number of recent scientific research papers; citing data that is unable to be replicated. The study reveals non-replicable data is cited 153 times more because the findings they lay out are deemed more interesting.

UCSD, “A New Replication Crisis: Research That Is Less Likely to Be True Is Cited More” at Neuroscience News

Of course. The papers that are unlikely to be replicated are mostly going to be stuff that people want and need to believe that isn’t necessarily so. Or not demonstrated via the sources that gave rise to the paper, anyway.

The paper is open access.

Papers in leading psychology, economic and science journals that fail to replicate and therefore are less likely to be true are often the most cited papers in academic research, according to a new study by the University of California San Diego’s Rady School of Management…

The paper reveals that findings from studies that cannot be verified when the experiments are repeated have a bigger influence over time. The unreliable research tends to be cited as if the results were true long after the publication failed to replicate.

“We also know that experts can predict well which papers will be replicated,” write the authors Marta Serra-Garcia, assistant professor of economics and strategy at the Rady School and Uri Gneezy, professor of behavioral economics also at the Rady School. “Given this prediction, we ask ‘why are non-replicable papers accepted for publication in the first place?’”

UCSD, “A New Replication Crisis: Research That Is Less Likely to Be True Is Cited More” at Neuroscience News

To begin any kind of serious analysis, we would need to classify the papers by general theme and general drift. That might give us a picture of what type of finding is too readily believed. But is it a picture anyone wants? Who, that has any say in the process, can really afford it?


You may also wish to read: Why it’s so hard to reform peer review Robert J. Marks: Reformers are battling numerical laws that govern how incentives work. Know your enemy!

2 Replies to “Big (?) Surprise: Cool, glitzy papers less likely to be replicated

  1. 1
    polistra says:

    Sucker Filter! Scammers use highly unlikely and absurd claims in their spam emails to avoid wasting time on non-gullible suckers. Successful scams require a lot of cultivation, bringing the sucker through a series of stages toward the payoff. Scammers save energy by rejecting poor prospects at the start.

    Wild claims don’t CREATE stronger belief; wild claims attract people who are inclined to believe wild claims.

  2. 2
    martin_r says:

    it seems that nothing changed since 2017 when the following was published:

    BBC:

    Most scientists ‘can’t replicate studies by their peers’

    Science is facing a “reproducibility crisis” where more than two-thirds of researchers have tried and failed to reproduce another scientist’s experiments, research suggests.

    https://www.bbc.com/news/science-environment-39054778

Leave a Reply