Culture Intelligent Design Naturalism Peer review Science

At PNAS: Reproducibility problems in science are slammed as fake news

Spread the love
File:FileStack.jpg
What’s hot? What’s not?/Niklas Bildhauer, Wikimedia

From Daniel Fanelli at PNAS:

Ultimately, the debate over the existence of a reproducibility crisis should have been closed by recent large-scale assessments of reproducibility. Their results, however, are either reassuring or inconclusive. A “Many labs” project reported that 10 of 13 studies taken from the psychological literature had been consistently replicated multiple times across different settings (21), whereas an analysis in experimental economics suggested that, of 18 studies, at least 11 had been successfully replicated (22). The largest reproducibility initiative to date suggested that in psychological science, reproducibility was below 50% (23). This latter estimate, however, is likely to be too pessimistic for at least two reasons. First because, once again, such a low level of reproducibility was not ubiquitous but varied depending on subfield, methodology, and expertise of the authors conducting the replication (23⇓–25). Second, and more importantly, because how reproducibility ought to be measured is the subject of a growing methodological and philosophical debate, and reanalyses of the data suggest that reproducibility in psychological science might be higher than originally claimed (23, 26, 27). Indeed, the very notion of “reproducible research” can be confusing, because its meaning and implications depend on what aspect of research is being examined: the reproducibility of research methods can in principle be expected to be 100%; but the reproducibility of results and inferences is likely to be lower and to vary across subfields and methodologies, for reasons that have nothing to do with questionable research and publication practices (28). More.

In short, it is a huge mess but can be defined out of existence.

Just dropping psychology from the list of sciences would be a big help. Then we would be dealing mainly with stuff like five-year survival rates and cancer treatments, which is what most people mean by science.

See also: Crisis in replication

One Reply to “At PNAS: Reproducibility problems in science are slammed as fake news

  1. 1
    asauber says:

    From the link:

    I argue that this crisis narrative is at least partially misguided.

    Picking and choosing our Crisis Narratives, are we?

    Recent evidence from metaresearch studies suggests…

    Well, that clinches it, doesn’t it?

    Wake me up when science stops insisting on being a joke.

    Andrew

Leave a Reply