She’s right to wonder: The research ones may not be telling us what we might think:
She points to one of the fundamental problems with fMRI: statistical excess:
The problem of statistical excess, called multiple comparisons, looms large over this part of the analysis… Multiple comparisons means too many statistical tests. The problem of multiple comparisons is like surveying 100,000 strangers about whether they know Beyoncé personally. None of those 100,000 people are actually acquainted with her, but for each person you ask, there is a 5 percent chance they will lie and say they are, just for kicks. In the end, you tally 5,000 friends of Beyoncé, even though the ground truth is that zero of those people are friends with her. If you had asked 100 strangers, you would only end up with five incorrect measurements, but because of sheer numbers and the probability of random deception, surveying 100,000 strangers results in 5,000 incorrect measurements.
KELSEY ICHIKAWA, “THE TROUBLE WITH BRAIN SCANS” AT NAUTILUS
fMRI data is often unreliable in this same way:
One person’s brain data has hundreds of thousands of voxels. By the sheer number of voxels and random noise, a researcher who performs a statistical test at every voxel will almost certainly find significant effects where there isn’t really one.
KELSEY ICHIKAWA, “THE TROUBLE WITH BRAIN SCANS” AT NAUTILUS
Ichikawa points to a famous fMRI study in which researchers, using standard statistical methods, found brain activity in a dead salmon.
Michael Egnor, “Why a budding neuroscientist is skeptical of brain scans” at Mind Matters News
Takehome: If people are judging what the rest of us think based on this, stop letting them. It could get beyond a joke.
Michael Egnor: After reading her perceptive essay about the problems in fMRI imaging in neuroscience, I’m sad that a gifted student has doubts about a career in the field. Neuroscience badly needs skeptics to show how unreliable technology, biased handling of data, and materialism’s conceptual mess frustrate science.
—
You may also wish to read: Why are some scientists turning away from brain scans? Sometimes, brain scans just sound like popular opinion. What’s wrong?
and
Brain scans can read your mind — in a dozen conflicting ways. A recent study involving 70 research groups identified sharp limitations in the value of brain imaging (fMRI) in understanding the mind.