More clutter building up in the inbox about the unreproducible results from social sciences: From ScienceDaily:
Today, in Nature Human Behavior, a collaborative team of five laboratories published the results of 21 high-powered replications of social science experiments originally published in Science and Nature, two of the most prestigious journals in science. They failed to replicate the results of more than a third of the studies and turned up significantly weaker evidence for the remainder compared to the original studies. Paper. (open access) – Colin F. Camerer, Anna Dreber, Felix Holzmeister, Teck-Hua Ho, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Gideon Nave, Brian A. Nosek, Thomas Pfeiffer, Adam Altmejd, Nick Buttrick, Taizan Chan, Yiling Chen, Eskil Forsell, Anup Gampa, Emma Heikensten, Lily Hummer, Taisuke Imai, Siri Isaksson, Dylan Manfredi, Julia Rose, Eric-Jan Wagenmakers, Hang Wu. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2018; DOI: 10.1038/s41562-018-0399-z More.
Some hint at the nature of the problem:
People who saw The Thinker, a sculpture by August Rodin, expressed more religious disbelief, Gervais reported in Science. And given all the evidence from his lab and others, he says there’s still reasonable evidence that underlying conclusion is true. But he recognizes the sculpture experiment was really quite weak.
“Our study, in hindsight, was outright silly,” says Gervais, who is now an assistant professor at the University of Kentucky.
A previous study also failed to replicate his experimental findings, so the new analysis is hardly a surprise. Richard Harris, “In Psychology And Other Social Sciences, Many Studies Fail The Reproducibility Test” at NPR
Some continue to hope that, despite overwhelming progressive bias, this time there will really be reform:
Still, the new study reveals a troubling aspect of successful experimental redos. Camerer’s team found that for repeat studies that panned out, which included four to five times as many participants as originally studied, the statistical strength to detect actual effects was weaker than reported for the initial investigations. In other words, the best replications — which exceeded initial studies in their ability to detect actual effects — were only partially successful.
One reason for that trend is that scientific journals have tended not to publish studies that disconfirm previous findings, leaving initial findings unchallenged until now, says study coauthor and psychologist Brian Nosek of the University of Virginia in Charlottesville. Even the most prestigious journals have often published results that garner lots of scientific and media attention but that could easily have occurred randomly, he says.
On the plus side, the new report appears as such practices are changing. “The social and behavioral sciences are in the midst of a reformation in scientific practices,” Nosek says. Bruce Bower, “‘Replication crisis’ spurs reforms in how science studies are done” at ScienceNews
Many of us have been watching such “reforms” parade by for two decades and nothing changes.
What if the drastic political skew toward progressivism is precisely social science’s value in the eyes of many progressive academics? Not only can’t the shoddy research be reformed but there is no true desire to reform it.
In other words, it’s not so much that we all think social psychology is really a science but that it is useful for many to pretend it is one.
If so, the scene raises an interesting question: The price these academics pay for being sure they will hear mostly what they want and expect to hear is that they won’t be hearing useful information but they can still call it science. With what outcome for themselves in the long run?
See also: Claimed link between creationism and “conspiracism”