Peer review

Even while prominent people in that field are coming to terms with the problem. From a New York Times science writer: It is one thing to frisk the studies appearing almost daily in journals that form the current back-and-forth of behavior research. It is somewhat different to call out experiments that became classics — and world-famous outside of psychology — because they dramatized something people recognized in themselves and in others. They live in the common culture as powerful metaphors, explanations for aspects of our behavior that we sense are true and that are captured somehow in a laboratory mini-drama constructed by an inventive researcher, or research team. Huh? Whether many people recognized something in themselves or not, the experiments Read More ›

It’s as if evolutionary biologists are beginning to take some of the problems of Darwinism seriously enough to discuss them openly, as failures in research. In this case, the failure of claims for sexual selection (females drive evolution by choosing the fittest mates) are openly publicized. In the past five years, meta-analyses and reviews have generated more evidence of bias in ecology and evolutionary biology research. For example, biases have been found in the literature on ideas such as feather color affecting mate choice in blue tits and black bib sizes indicating male dominance in house sparrows. As with zebra finch leg bands, such biases don’t necessarily invalidate the hypotheses themselves, but undermine the strength of evidence for them, leaving Read More ›

From Dalmeet Singh Chawla at Undark: N SEPTEMBER 2016, the psychologist Dana Carney came forward with a confession: She no longer believed the findings of a high-profile study she co-authored in 2010 to be true. The study was about “power-posing” — a theory suggesting that powerful stances can psychologically and physiologically help one when under high-pressure situations. Carney’s co-author, Amy Cuddy, a psychologist at Harvard University, had earned much fame from power poses, and her 2012 TED talk on the topic is the second most watched talk of all time. Carney, now based at the University of California, Berkeley, had, however, changed her mind. “I do not believe that ‘power pose’ effects are real,” she wrote on her website in Read More ›

While we are talking about retractions, from Nathan Rosenberg and Bryce Wilson Stucki at the New Food Economy: On Friday, the Centers for Disease Control and Prevention (CDC) confirmed that a widely cited result on farmer suicides was wrong. Over the past several months, numerous writers and reporters have relied on the finding, originally from a 2016 CDC report, to argue that farmers have the highest suicide rate in the country. That report found that workers in the “farming, fishing, and forestry,” job category killed themselves at over four times the national average, far and away the highest in the study. But on Friday, Courtney Lenard, a public relations official, confirmed to us in an email that CDC had misclassified Read More ›

From Alison McCook at Retraction Watch: n a new preprint posted to bioRxiv, image sleuths scanned hundreds of papers published over a seven-year period in Molecular and Cellular Biology (MCB), published by the American Society for Microbiology (ASM). The researchers — Arturo Casadevall of Johns Hopkins University, Elisabeth Bik of uBiome, Ferric Fang of the University of Washington (also on the board of directors of our parent non-profit organization), Roger Davis of the University of Massachusetts (and former MCB editor), and Amy Kullas, ASM’s publication ethics manager — found 59 potentially problematic papers, of which five were retracted. Extrapolating from these findings and those of another paper that scanned duplication rates, the researchers propose that tens of thousands of papers Read More ›

In a world where some researchers earnestly study the question of why so many people don’t ”trust science,” we learn from Michael Marlow & Edward Archer at RealClearScience: In contrast [to confidence in politicians], public confidence in the ‘scientific community’ runs at 40% and has remained stable since the 1970s. This trust, however, turns out to be seriously misplaced when it comes to the government’s data on what we eat and drink. The nutrition research methods of the Centers for Disease Control and Prevention (CDC) and the US Department of Agriculture (USDA) are based on the naïve but politically expedient notion that a person’s usual diet can be measured simply by asking what he or she remembered eating and drinking. Read More ›

From Ben Blum at Medium: Whether you learned about Philip Zimbardo’s famous “Stanford Prison Experiment” [1973] in an introductory psych class or just absorbed it from the cultural ether, you’ve probably heard the basic story. Zimbardo, a young Stanford psychology professor, built a mock jail in the basement of Jordan Hall and stocked it with nine “prisoners,” and nine “guards,” all male, college-age respondents to a newspaper ad who were assigned their roles at random and paid a generous daily wage to participate. The senior prison “staff” consisted of Zimbardo himself and a handful of his students. The study was supposed to last for two weeks, but after Zimbardo’s girlfriend stopped by six days in and witnessed the conditions in Read More ›

At PhilEvents: Recent epistemology has seen an explosion of interest in disagreement and other related questions in social epistemology. While much progress has been made on abstract and general epistemological issues relating to disagreement, there has been surprisingly little discussion of how, if at all, these lessons can be applied to disagreement within science in particular. Furthermore, several aspects of the topic go beyond merely applying lessons from analytic epistemology. For example, scientific disagreement is unlike many ordinary cases of disagreement in that there is often little reason to think that the disagreement is due to a simple mistake by one of the parties of the type often appealed to in the epistemology of disagreement literature. Rather, if there is Read More ›

From bioegineer and editor Nicolai Slavov at The Scientist: Have you read a paper and thought: “How could peer reviews support the publication of such a paper?” I have. More than once. Other times, I have read fascinating papers outside of my field and wondered what the concerns of the experts who peer reviewed the study were. What important caveats am I missing? Sometimes, I am lucky and find the answers to such questions: A few publications, including those from EMBO Press and eLife, publish the peer reviews alongside the papers. Reading such peer reviews has provided an additional dimension of appreciating and understanding the experiments and the findings, especially when I am not very familiar with the topic. But Read More ›

If science were mostly disputes over trivia, replication would not matter. But when studies of the effectiveness of cancer treatment fail replication,  you might want to take an interest in this problem if you think you might ever need cancer treatment. After all the hoopla about replication studies as one plank in the reform of a broken peer review system, it’s interesting to see how the top science journal reacted. In the first installment by Mante Nieuwland over at Retraction Watch, we learned, Importantly, our multi-laboratory replication study tackled all the methodological and statistical issues with DUK05 that have come up in recent years. We tested a sample more than 10 times greater than that of DUK05, we employed both Read More ›

From Mante Nieuwland at Retraction Watch: On April 10th 2018, eLife published the first large-scale direct replication study in the field of cognitive neuroscience, co-authored by 22 colleagues and myself. This publication detailed a replication effort that spanned 9 laboratories and attempted to replicate a high-impact 2005 publication in the prestigious journal Nature Neuroscience from DeLong, Urbach and Kutas (from hereon referred to as DUK05). People often ask why our replication study was not published in Nature Neuroscience, especially in light of its recent public commitments to replication research (here and here). It certainly wasn’t for our lack of trying. In this post, I offer a behind-the-scenes account of what happened when we tried to replicate DUK05 and submitted our Read More ›

From Daniel Muth at Living Church, reviewing Steve Meyer’s Darwin’s Doubt: I am fairly certain that there are thoughtful and potentially influential intellectual movements that have been subjected to more shameful and inexcusable misrepresentation and ill treatment than Intelligent Design (ID), but the list is not long (Roman Catholic teaching on artificial birth control comes to mind). To be fair, ID theorists have invited critique in no small part by tending to hold theirs out as a valid area of scientific research while mainly publishing popular books rather than peer-reviewed articles. If their intention was not to be lumped in with creationists, it has not worked. From the disastrous Dover School Board lawsuit to the propaganda screeds of the New Read More ›

A handy guide from Tasnim Elmamoun at PLOS blogs: So… how do we identify spin? As Chiu and colleagues point out, spin can take a variety of forms, including: 1) “inappropriate study given study data;” 2) “inappropriate extrapolations or recommendations for clinical practice;” 3) “selective reporting;” 4) and “more robust or favorable data presentation.” Let’s unpack these a little bit. The first of these types, “inappropriate study given study data,” occurs when findings simply are interpreted incorrectly. Specifically, Chiu et al. found that this type of spin is commonly used in conjunction with casual (or colloquial) language, which in many cases has the potential to alter interpretation of the data. So how do researchers strike a balance between using scientific Read More ›

From Skye C. Cleary and Massimo Pigliucci at Aeon: A strange thing is happening in modern philosophy: many philosophers don’t seem to believe that there is such a thing as human nature. What makes this strange is that, not only does the new attitude run counter to much of the history of philosophy, but – despite loud claims to the contrary – it also goes against the findings of modern science. This has serious consequences, ranging from the way in which we see ourselves and our place in the cosmos to what sort of philosophy of life we might adopt. Our aim here is to discuss the issue of human nature in light of contemporary biology, and then explore how Read More ›

Report: NAS Launches New Report: “The Irreproducibility Crisis of Modern Science”: Today is the launch of NAS’s newest report by David Randall and Christopher Welser. The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform examines the different aspects of the reproducibility crisis of modern science. Our goal is to bring the reproducibility crisis to the forefront of public awareness and to call on policymakers to take effective steps to address it. We also include a series of policy recommendations, scientific and political, for alleviating the reproducibility crisis. NAS was founded on, and continues to be guided by the idea that the pursuit of truth is the highest purpose of scholarly work. Civil and open debate is Read More ›