Uncommon Descent Serving The Intelligent Design Community

P threshold values often likely false?

Denyse O'Leary
April 4, 2018
Peer review
Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

From John Ioannidis at the Journal of the American Medical Association:

vPalues and accompanying methods of statistical significance testing are creating challenges in biomedical science and other disciplines. The vast majority (96%) of articles that report P values in the abstract, full text, or both include some values of .05 or less. However, many of the claims that these reports highlight are likely false. Recognizing the major importance of the statistical significance conundrum, the American Statistical Association (ASA) published3 a statement on P values in 2016. More.

See also: P-values: Scientists slam proposal to raise threshold for statistically significant findings

and

Ioannidis again on misleading meta-analyses

Comments
Bob O'H: I work with statistics in medicine, and of course I agree with you. It's true that there is a lot of abuse of statistics, and the main reason is that almost no one understands or know it, but practically everybody has to use it. Regarding p values, my idea is simple: it's note a problem of setting thresholds, o.o5 or o.o1. The problem is that there should be no absolute thersholds, but simply a credible assessment of the results, case by case. It has been my experience that in some cases, where the results were significant at a level of <10^-16 (the lowest value computed by R), a reviewer asked that the p value were reported simply as <0.01. That's, IMO, simple folly. We need to understand what a p value means in its specific contex, and not just classify the results as "significant" or "not significant".gpuccio
April 6, 2018
April
04
Apr
6
06
2018
03:45 AM
3
03
45
AM
PDT
polistra - OK, I'm a statistician so I might be biased, but I don't think you appreciate the impossibility of doing science without statistics. For example, how would you know if a new drug worked or was actually harmful? How would you design an experiment to see if a drug could cure the common cold? Remember that the severity of a cold varies, and this depends on, amongst other things, the age and health of the person infected)? Also remember that any bad side effects are unlikely to show themselves in all patients.Bob O'H
April 6, 2018
April
04
Apr
6
06
2018
12:05 AM
12
12
05
AM
PDT
Just get rid of stats. A clear and meaningful observation is clear and meaningful by itself, without stats. If you can't set up an experiment or observation that yields a clear decision without stats, you still have a useful result. The fact that you CAN'T set up such an experiment IS a useful piece of information and deserves to be reported. If you have to use stats of any kind, you're fudging and faking by definition.polistra
April 5, 2018
April
04
Apr
5
05
2018
01:43 AM
1
01
43
AM
PDT
The first link should be https://jamanetwork.com/journals/jama/fullarticle/2676503Bob O'H
April 4, 2018
April
04
Apr
4
04
2018
08:41 AM
8
08
41
AM
PDT

Leave a Reply