News Peer review Psychology

Is the scientific mood souring?

Spread the love

UCLA psychologist Matthew D. Lieberman thinks so, at The Edge, a sciencey thinksite:

When I got into the field it didn’t seem like there were any career-threatening giant debates going on, and now it just seems to be all over the place. Every 20 to 30 years my field of social psychology seems to go through another crisis. There was a crisis in the fifties, where somebody published a paper and it killed the most exciting area of research in social psychology for 20 years.

Then in the seventies there was something actually called “the crisis of social psychology”. Now there’s “the replication crisis”, which is a replication crisis in science, if it’s even a crisis. It’s just that we need to be reminded sometimes that when you see the first flashy study published in science or psych science, it’s just an anecdote. It’s a scientific anecdote, and we should go collect some more. It can be a really exciting one that you want to tell all your friends about, but it’s one little tiny piece of data. We’ve perhaps taken to assuming those things were facts, and then we’re shocked when those things don’t replicate in study number two.

There’s a lot of stuff going on where there’s now people making their careers out of trying to take down other people’s careers. The replication isn’t necessarily an unbiased process, as it’s presented. There are camps, and suddenly now failing to replicate someone else is really seen as an indictment, in many people’s eyes, of the person who did the original research, rather than saying there’s expectancy effects. If I expect not to replicate someone’s work, that’s going to influence how I design my study, the measures I look at, it’s going to influence how I interact with my participants. I’ve heard stories about participants saying that they’ve been told, “Oh, you’re just in a replication effort, so it doesn’t matter if you know more than you should.” There are things going on, and it is troubling to me.

Chickens. Home. Roost. Over at Discover, Neuroskeptic responds to what is clearly an attempt to quiet the growing dissent over imaginative/imaginary social science findings:

… all I’m saying is that there’s something odd about the idea that ones qualifications should include a track record in finding positive results in the field in question. That seems to be putting the cart before the horse. I agree that replicators should have the necessary technical skills, but I question whether generating positive (as opposed to negative) results can be used as a proxy for being skilled.

That would make sense if we assume that our basic psychological theory (e.g. of social priming) is valid, and therefore that at least some of our effects are real and replicable. If we grant that, then yes, we could assume that people who fail to find effects, must be doing it wrong. (If magic exists, then non-wizards are muggles.)

But can we assume that? Isn’t that, in fact, the issue under debate in many cases?

Actually, the really big issue here is much simpler. Social psychologists are overwhelmingly progressives (or if you like, leftists, or less plausibly, liberals), to a statistically impossible lack of diversity, compared to the public. A suspiciously large number of the issues I hear about stem from conflicts between the data and what they “know” to be true (flyover country is racist, religious people have poorer physical and mental health, etc.). I predict the scandals will continue. You can lead a horse to water but you can’t make him think. – O’Leary for News

Hat tip: Stephanie West Allen at Brains on Purpose

One Reply to “Is the scientific mood souring?

  1. 1
    StephenA says:

    This is massively off topic, but it seems as good a place as any to have my rant/lecture.

    A Practical Application of ID theory:

    First, a bit of context. I am a video game developer. I recently attended PAX Dev, a conference where fellow developers can come together and share insights about our work. One of the talks I attended was about Procedural Generation in games.
    For those of you who don’t know, Procedural Generation is when a developer creates an algorithm that will randomly assemble some aspect of the game (most commonly the levels or world that the game takes place in).
    The speaker noted that if the players know that a level is Procedurally Generated they tend to stop assigning signifigance events that happen in the game. For instance, if a player is is walking along a path and lightning strikes a tree nearby, they will assume it was just a random event that they happened to see. Whereas a player that knows the game world was designed will begin to wonder what the intent of the developer is. They will become intruiged, which is very valuable to us as game developers.

    The speaker also noted that even though procedurally generated levels in theory can have astronomical variablity, in practice they soon start feeling like they are all much the same. This is something that my own experiences using procedural generation had made me aware of, but I had never thought to apply ID theory to the problem.

    ID Theory states that information (or at least, particular kinds of information) can never be created without intelligent intervention. NFL Theorems indicate that you cannot get more information out of a search algorithm than is in the algorithm in the first place. (correct me if I have misstated anything)
    If we think of the level generating algorithms as searches across probability space, it quickly becomes apparent what the problem is. No new information is being added. Since we can’t get any more information out of a search than we put in, even if we run it multiple times, it means that each variation of the level is being built with much the same information. And players, being human, quickly notice that they aren’t getting any new information.

    No one I know has offered a complete solution, but ID tells us what is not a solution – making better algorithms. Algorithms can never produce the new information that human players crave. The most they might do is store up more information and then release it slowly.

Leave a Reply