That’s a big part of the problem with peer review and replication failures. Don’t believe me? Get this:
In response to massive failures in so-called social sciences:
“The findings reinforce the roles that two inherent intuitions play in scientific decision-making: our drive to create a coherent narrative from new data regardless of its quality or relevance, and our inclination to seek patterns in data whether they exist or not,” he says.
Dingledine also says the results speak to a bigger problem, something Kahneman famously described in an open letter to colleagues in 2012 as a “train wreck looming”: the widespread failure to replicate the findings of many important studies in the social sciences.
That wreck may well be upon us. Paul Biegler, “Even scientists jump to conclusions – and that’s a problem” at Cosmos Magazine
Failure to replicate is everywhere in the social sciences, as Biegler’s article shows. Will it get so bad that a given replication could just be chance? He adds, “Dingledine, however, suggests the reproducibility crisis must expand its focus from experimental design to include the influence of “human nature” on scientific judgment, which he says has not received due attention.”
What? “human nature” intervened? Along with the article’s title, “Even scientists jump to conclusions,” the way this problem is described implies that scientists are somewhat superior to the rest of us but just not living up to their superiority.
Okay. Now we at least know what the first thing needed is: Get off that high horse.
See also: Why, in many cases, you’d be a fool to “trust science” If you also think that data is a source of information, that is. And also have to live in the real world.
Back to school briefing: Seven myths of social psychology: Many lecture room icons from decades past are looking tarnished now. (That was 2014 and it has gotten worse since.)
Backing up the particle physicist who says there is “baked in” bias in science Kirk Durston: If anyone is interested in a list of references with links, backing up the serious problem that science is facing right now, I wrote a blog post a while back that has a “Further Reading” section at the bottom of it. It currently stands at 47 links, counting Sabine Hossenfelder’s latest blog post.
and
Particle physicist: Science is suffering from “baked in” bias The challenge is simpler than sometimes supposed. People must be willing to accept a truth they don’t like. If the universe is not as we would like it to be, imagining a different one is fun and maybe profitable, maybe aesthetically pleasing. But it is not science.