Uncommon Descent Serving The Intelligent Design Community

Another nugget from the quote mine: In evolutionary biology, “almost no findings are replicated”


Jerry Coyne is always fun. He has the distinction of being a Darwinist who is perfectly honest about the war between Darwinism and any belief in the uniqueness of humans – many examples here, and such relief from any contact with Christian Darwinists.

Recently, he commented on an article in The New Yorker by Jonah Lehrer, “The truth wears off: is there something wrong with the scientific method?”.

Basically, Lehrer says, an initial demonstration in science tends to weaken or disappear when attempts are made to replicate it:

On September 18, 2007, a few dozen neuroscientists, psychiatrists, and drug-company executives gathered in a hotel conference room in Brussels to hear some startling news. It had to do with a class of drugs known as atypical or second-generation antipsychotics, which came on the market in the early nineties. The therapeutic power of the drugs appeared to be steadily falling. A recent study showed an effect that was less than half of that documented in the first trials, in the early nineties. Before the effectiveness of a drug can be confirmed, it must be tested again and again. The test of replicability, as it’s known, is the foundation of modern research. It’s a safeguard for the creep of subjectivity. But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts are losing their truth.Read more here [some more there, but you must pay for the rest].

Coyne writes in “The ‘decline effect’: can we demonstrate anything in science?”

I tend to agree with Lehrer about studies in my own field of evolutionary biology. Almost no findings are replicated, there’s a premium on publishing positive results, and, unlike some other areas, findings in evolutionary biology don’t necessarily build on each other: workers usually don’t have to repeat other people’s work as a basis for their own. (I’m speaking here mostly of experimental work, not things like studies of transitional fossils.) Ditto for ecology. Yet that doesn’t mean that everything is arbitrary. I’m pretty sure, for instance, that the reason why male interspecific hybrids in Drosophila are sterile while females aren’t (“Haldane’s rule”) reflects genes whose effects on hybrid sterility are recessive. That’s been demonstrated by several workers. And I’m even more sure that humans are more closely related to chimps than to orangutans. Nevertheless, when a single new finding appears, I often find myself wondering if it would stand up if somebody repeated the study, or did it in another species.

Good thing to wonder about. Time more people wondered about that. Breath of fresh air.

Personally, I am most wary of any finding that is breathlessly touted as proving what our moral and intellectual superiors (in their own view) totally believe already, so why anyone even did the study isn’t clear. Couldn’t they just save a bundle by making the whole thing up? Given that we all must swallow it anyway, or so we are told.

Uh-huh. Here’s the deal: the effect doesn’t “wear off.” It was never there in the first place—or at least not at the magnitude claimed in the trial reports. There are countless examples in the industry. First, there is a substantial, documented placebo effect in drug trials. But it’s not just the patients who perceive efficacy in non-efficacious drugs. The placebo effect extends to the physicians conducting the study and the statisticians evaluating the data. The fact is that investigators often have a stake in the success of the trial medication, and not just a financial one. The more hype surrounding the drug, the more this is true. Think of the SSRIs in the 90’s. The media story line was that modern medicine had conquered depression with a simple pill. This story was irresistible on many levels to the cultural vanguard. Even with all this hype and wishful thinking, the performance of the SSRIs in early clinical trials was in fact middling. The only reason the first compounds made it to market was that the drug companies downplayed the neutral trials. So you have physicians conducting the studies who very much want the results to be positive. Then you have the statisticians. Sorry, but it is a simple fact that they can cut the data many different ways. I have been in meetings where the statistician literally asked what result was desired. And this is an industry that attempts to keep itself honest by requiring double-blind trials! I have to laugh when I see unblinded “scientific studies” reported in the press that are supposed to support evolution or some other cause de jour. None of these studies would ever be admitted as anything other than conditional evidence by the FDA. And no, it doesn’t matter if it’s published in “Science” or “Nature.” The glowing SSRI studies were published in tony medical journals, too. Peer review is compromised when all of the peers think the same way. They will be too forgiving of results that are politically correct and too dismissive of results that go against the conventional wisdom. allanius
This is always the result when government is involved. When someone in government, I think it was NHTSA, came up with the idea of the "third brakelight" and they went forward with testing, was there ever wny doubt about the outcome of the testing, and how additional manufacturing expense was assured? Try bringing an auto that you bought in another country, here and see if you can register it. The absence of the third brakelight and other stuff is why you may as well forget it. groovamos

Leave a Reply