News Peer review

Why ask nature questions when we don’t want her answers?

Spread the love

We don’t often say “be sure to read this article,” but be sure to read this article by Jerry Adler in Pacific Standard, on the background to many recent instances of research misconduct.

Something unprecedented has occurred in the last couple of decades in the social sciences. Overlaid on the usual academic incentives of tenure, advancement, grants, and prizes are the glittering rewards of celebrity, best-selling books, magazine profiles, TED talks, and TV appearances. A whole industry has grown up around marketing the surprising-yet-oddly-intuitive findings of social psychology, behavioral economics, and related fields. The success of authors who popularize academic work—Malcolm Gladwell, the Freakonomics guys, and the now-disgraced Jonah Lehrer—has stoked an enormous appetite for usable wisdom from the social sciences. And the whole ecosystem feeds on new, dramatic findings from the lab. “We are living in an age that glorifies the single study,” says Nina Strohminger, a Duke post-doc in social psychology. “It’s a folly perpetuated not just by scientists, but by academic journals, the media, granting agencies—we’re all complicit in this hunger for fast, definitive answers.”

Yes, and—though Adler never quite gets here—very often, complicit in the hunger for answers that confirm what people think they already know. Not only that, an honest study that came up with genuinely politically incorrect findings might be banished on principle, along with the researcher.

The researcher is supposed to realize that only politically correct findings are part of the quest for knowledge.

Adler’s article addresses how p-values are misused to produce the desired results:

P is a central concept in statistics: It’s the mathematical factor that mediates between what happens in the laboratory and what happens in the real world. The most common form of statistical analysis proceeds by a kind of backwards logic: Technically, the researcher is trying to disprove the “null hypothesis,” the assumption that the condition under investigation actually makes no difference. In Sanna’s experiment, the null hypothesis is that elevation has no effect on how much hot sauce people dole out. If that is actually what the data shows, then the experiment is over, the null hypothesis wins, and the researcher can forget about going on The Daily Show. But in practice things usually aren’t quite as clear-cut.

In psychology, p value is less than .05 or five percent. For the Higgs boson, it was one in 3 million.

As Simmons showed, psychologists who deploy enough statistical sleight of hand can find “significance” in almost any data set. How often do researchers give in to this temptation? One way to roughly answer that question would be to study the distribution of p values over a large sample of papers. If researchers are fiddling with the math to get their results just under the 0.05 threshold, then you might expect to see a cluster of values just below 0.05, rather than the more normal distribution that might have arisen as a result of chance. In an analysis of a year’s worth of papers in three leading psychology journals, two researchers found “a peculiar prevalence of p values just below 0.05.”

Adler discusses current attempts at reform, and one can only wish the reformers well. The difficulty is that in a world where nature may or may not co-operate with the “correct” results, the problems may well lie deeper than the intended reforms.

Hat tip: Stephanie West Allen at Brains on Purpose

Follow UD News at Twitter!

4 Replies to “Why ask nature questions when we don’t want her answers?

  1. 1
    bornagain77 says:

    It is amazing that many scientists today think nothing of seeking truth in a philosophy (materialism) that cannot supply a foundation for truth:

    Philosophical materialism, the foundation of Western academia, is showing ever-widening cracks.
    Excerpt: The philosophical materialist says “truth can only be found by the application of reason and laws of logic to empirical experience”. But he has no way to validate the very statement he has just made.,,, He is in no better shape than one who says “there is no absolute truth except the absolute truth that there is no absolute truth”.
    But to the theist (and especially to the Christian) the existence of absolute truth is not an issue. It is not an issue because the supreme personal God he believes in is the source of all absolute truth. This includes all metaphysical truth as well as the universal invariant laws of logic required to do science.
    http://www.csulb.edu/~mbiedeba/ch1.html

    Exactly why should the atheist presuppose there is a meaningful truth to be discovered if he truly believes we are the result of a meaningless. ‘random’, process?

    “Consequently atheism turns out to be too simple. If the whole universe has no meaning, we should never have found out that it has no meaning: just as, if there were no light in the universe and therefore no creatures with eyes, we should never know it was dark. Dark would be a word without meaning.”
    CS Lewis – Mere Christianity

    Around the 13:20 minute mark of the following video Pastor Joe Boot comments on the self-defeating nature inherent in the atheistic worldview in regards to seeking absolute truth:

    Defending the Christian Faith – Pastor Joe Boot – video
    http://www.youtube.com/watch?v=wqE5_ZOAnKo

    “If you have no God, then you have no design plan for the universe. You have no prexisting structure to the universe.,, As the ancient Greeks held, like Democritus and others, the universe is flux. It’s just matter in motion. Now on that basis all you are confronted with is innumerable brute facts that are unrelated pieces of data. They have no meaningful connection to each other because there is no overall structure. There’s no design plan. It’s like my kids do ‘join the dots’ puzzles. It’s just dots, but when you join the dots there is a structure, and a picture emerges. Well, the atheists is without that (final picture). There is no preestablished pattern (to connect the facts given atheism).”
    Pastor Joe Boot

    And yet even though materialism cannot ground purpose, meaning, and most importantly, truth, never-the-less atheists, even as they deny it, must sneak teleology into the scheme somehow so as to be able to do science:

    Design Thinking Is Hardwired in the Human Brain. How Come? – October 17, 2012
    Excerpt: “Even Professional Scientists Are Compelled to See Purpose in Nature, Psychologists Find.” The article describes a test by Boston University’s psychology department, in which researchers found that “despite years of scientific training, even professional chemists, geologists, and physicists from major universities such as Harvard, MIT, and Yale cannot escape a deep-seated belief that natural phenomena exist for a purpose” ,,,
    Most interesting, though, are the questions begged by this research. One is whether it is even possible to purge teleology from explanation.
    http://www.evolutionnews.org/2.....65381.html

    It is interesting to note that the very belief that there is some type of unity, some purpose, some overriding connection of the laws of physics, i.e. a theory of everything, is itself a belief that arises from the presupposition of Design in the universe. ,,, A Theistic presupposition.

    In Cambridge, Professor Steve Fuller discusses intelligent design – Video
    http://www.uncommondescent.com.....nd-others/
    At 17:34 minute mark of the video, Dr. Steve Fuller states:
    “So you think of physics in search of a “Grand Unified Theory of Everything”, Why should we even think there is such a thing? Why should we think there is some ultimate level of resolution? Right? It is part, it is a consequence of believing in some kind of design. Right? And there is some sense in which that however mulrifarious and diverse the phenomena of nature are, they are ultimately unified by the minimal set of laws and principles possible. In so far as science continues to operate with that assumption, there is a presupposition of design that is motivating the scientific process. Because it would be perfectly easy,, to stop the pursuit of science at much lower levels. You know understand a certain range of phenomena in a way that is appropiate to deal with that phenomena and just stop there and not go any deeper or any farther.”,,, You see, there is sense in which there is design at the ultimate level, the ultimate teleology you might say, which provides the ultimate closure,,”

    Verse and Music:

    2 Timothy 3:7
    always learning but never able to come to a knowledge of the truth.

    Shatter Me Featuring Lzzy Hale – Lindsey Stirling
    https://www.youtube.com/watch?v=49tpIMDy9BE

  2. 2
    Barb says:

    Malcolm Gladwell’s problem is oversimplification of complex issues (http://www.cjr.org/the_observa.....eaches.php). But his books are entertaining (I’ve perused “Blink” and “Outliers”).

  3. 3
    CuriousCat says:

    This may be too much of a technical note of the use (or misuse) of P-value in different branches of science, nevertheless I’ll mention it. P-value is not the probability of null hypothesis being correct given the current data. It is the probability of obtaining the current data given that null hypothesis is correct. Hence, P-value is NOT equal to P(H0 true|data obtained), but P-value IS equal to P(data obtained|H0 is true). Experimenters in psychology and (in many cases) biology misuse (or misinterpret) this subtlety in the meaning of P-value, and take a P-value smaller than 0.05 as an indication of the improbability of the null hypothesis being true and reject it in favor of the alternative hypothesis. Since it is usually the alternative hypothesis that draws attention in the scientific community and makes the research publishable, researches collect many data points and “filter” them to get a magical P-value smaller than 0.05!

    A detailed discussion can be found here:
    http://ist-socrates.berkeley.e.....Cohen1.pdf

  4. 4
    Robert Byers says:

    its people like this adler that have been the problem too.
    Whats wrong with TED talks? They are better then writers on science but who don’t do anything like Adler.
    Its all been this way forever.
    Its just people trying to figure out things. they are right or wrong and smart and dumb.
    Science methosology hardly helps push forward good ideas and hardly helps stop bad ones.
    Evolution is case in point.

Leave a Reply