From the Weekly Standard:
The behavioral sciences scandal
On this August morning Science magazine had published a scandalous article. The subject was the practice of behavioral psychology. Behavioral psychology is a wellspring of modern journalism. It is the source for most of those thrilling studies that keep reporters like Vedantam in business.
Over 270 researchers, working as the Reproducibility Project, had gathered 100 studies from three of the most prestigious journals in the field of social psychology. Then they set about to redo the experiments and see if they could get the same results. Mostly they used the materials and methods the original researchers had used. Direct replications are seldom attempted in the social sciences, even though the ability to repeat an experiment and get the same findings is supposed to be a cornerstone of scientific knowledge. It’s the way to separate real information from flukes and anomalies.
And guess what happened when they tried? Aw come on, don’t be shy, take a guess.
The basic thesis of many exploded papers, in shorthand, is flyover country is racist. Or something similar. Bad science but good copy.
A huge underlying problem:
The defenders of behavioral science like to say it is the study of “real people in real-life situations.” In fact, for the most part, it is the study of American college kids sitting in psych labs. And the participation of such subjects is complicated from the start: The undergrads agree to become experiment fodder because they are paid to do so or because they’re rewarded with course credit. Either way, they do what they do for personal gain of some kind, injecting a set of motivations into the lab that make generalizing even riskier
Behind the people being experimented upon are the people doing the experimenting, the behavioral scientists themselves. In important ways they are remarkably monochromatic. We don’t need to belabor the point. In a survey of the membership of the Society for Personality and Social Psychology, 85 percent of respondents called themselves liberal, 6 percent conservative, 9 percent moderate. Two percent of graduate students and postdocs called themselves conservative. “The field is shifting leftward,” wrote one team of social psychologists (identifying themselves as “one liberal, one centrist, two libertarians, two who reject characterization,” and no conservatives). “And there are hardly any conservative students in the pipeline.” A more recent survey of over 300 members of another group of experimental psychologists found 4 who voted for Mitt Romney.
The self-correction essential to science is less likely to happen among people whose political and cultural views are so uniform. This is especially true when so many of them specialize in studying political and cultural behavior. Their biases are likely to be invisible to themselves and their colleagues. More.
All the signs of serious trouble have been there for decades, scrupulously ignored. Wait till you read about the “social priming” circus.
Incidentally,
Even before the Reproducibility Project, direct replications failed to find evidence for many other effects that the social psychology literature treats as settled science. “Single-exposure conditioning”?—?if you’re offered a pen while your favorite music is playing, you’ll like the pen better than one offered while less appealing music plays. The “primacy of warmth effect,” which tells us our perceptions are more favorable to people described as “warm” than to people described as “competent.” The “Romeo and Juliet effect”: Intervention by parents in a child’s romantic relationship only intensifies the feelings of romance. None of these could be directly replicated.
Perhaps most consequentially, replications failed to validate many uses of the Implicit Association Test, which is the most popular research tool in social psychology. Its designers say the test detects unconscious biases, including racial biases, that persistently drive human behavior. Sifting data from the IAT, social scientists tell us that at least 75 percent of white Americans are racist, whether they know it or not, even when they publicly disavow racial bigotry. This implicit racism induces racist behavior as surely as explicit racism. The paper introducing the IAT’s application to racial attitudes has been cited in more than 6,600 studies, according to Google Scholar. The test is commonly used in courts and classrooms across the country.
That the United States is in the grip of an epidemic of implicit racism is simply taken for granted by social psychologists?—?another settled fact too good to check. Few of them have ever returned to the original data. Those who have done so have discovered that the direct evidence linking IAT results to specific behavior is in fact negligible, with small samples and weak effects that have seldom if ever been replicated. More.
Vote for it, pay for it, learn to love it, I guess.
See also: Japanese universities shedding liberal arts departments (biggest Sokal hoax in the universe considered superfluous)
Follow UD News at Twitter!