Bayesian statistics are used, for example, in spam filter technology, identifying probable spam by examining vast masses of previous messages:
Daniel Díaz: There has been a long, long debate between two different approaches to statistics [Bayesian and frequentist]. I can see many cases in which one of the approaches is better than the other. So it depends on the problem that you are working on.
Robert J. Marks: As an engineer I’m always interested in reduction to practice. And one of the things that Bayesian statistics is used for is spam filtering. You gather a lot of emails and figure out the probability that if a “Nigerian prince” is mentioned, it is spam. A lot of people fell for it. … But you can look at past data. You can look at all of the labeled emails that have been labeled spam and figure out how many that had “Nigerian prince” in them were actually spam.
The spam filter is much more complicated than this but that’s a simple illustration. And this is the reason we use Bayesian statistics … Your Bayesian statistics can say, what is the probability that this email is spam? As an engineer, I always say that reduction to practice is the proof of the validity of a theory. So I think that the criticism of Bayesian statistics that I mentioned was totally inaccurate. It’s not a good argument.
Daniel Díaz: It is also useful in some areas of medicine. I work in biostatistics, so I know that in many, many analysis of treatments, Bayesian results are very useful and more useful in some cases than the frequentist approach. But as I say, there are other approaches, there are other problems in which the frequentist approach works better. The important thing for our conversation is that, actually, the right way to approach our problem is using Bayesian theory and Bayes’ theorem in order to determine the distribution of maximum entropy to use. So for fine-tuning, in order to avoid certain problems in the past, the right approach is to consider that prior distribution was given in terms of the maximum entropy. And that is also done with the help of Bayes’ theorem.
Ola Hössjer: Yes. I totally agree with that because we use Bayesian statistics and put a prior distribution on the possible values or a certain constant or nature. Because of the Bayesian statistics approach, not the frequentist approach, we are in fact, able to talk about the probability of this interval. With a frequentist approach that would not have been possible. We could only talk about how consistent each possible constant of nature is with data and so on.News, “Fine-tuning? How Bayesian statistics could help break a deadlock” at Mind Matters News
Takehome: The frequentist approach assesses the probability of future events but the Bayesian approach assesses the probability of events that have already occurred.
You may also wish to read/hear the first portion of the episode:
Ours is a finely tuned — and No Free Lunch — universe. Mathematician Ola Hössjer and biostatistician Daniel Andrés Díaz-Pachón explain to Walter Bradley Center director Robert J. Marks why nature works so seamlessly. A “life-permitting interval” makes it all possible — but is that really an accident?