New Dembski-Marks Paper

_{William Dembski

December 7, 2009

Intelligent Design}

Share: Facebook; Twitter; LinkedIn; Flipboard; Print; Email

William A. Dembski and Robert J. Marks II, “Bernoulli’s Principle of Insufficient Reason and Conservation of Information in Computer Search,” Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics. San Antonio, TX, USA – October 2009, pp. 2647-2652.

Abstract: Conservation of information (COI) popularized by the no free lunch theorem is a great leveler of search algorithms, showing that on average no search outperforms any other. Yet in practice some searches appear to outperform others. In consequence, some have questioned the significance of COI to the performance of search algorithms. An underlying foundation of COI is Bernoulli’s Principle of Insufficient Reason1(PrOIR) which imposes of a uniform distribution on a search space in the absence of all prior knowledge about the search target or the search space structure. The assumption is conserved under mapping. If the probability of finding a target in a search space is p, then the problem of finding the target in any subset of the search space is p. More generally, all some-to-many mappings of a uniform search space result in a new search space where the chance of doing better than p is 50-50. Consequently the chance of doing worse is 50-50. This result can be viewed as a confirming property of COI. To properly assess the significance of the COI for search, one must completely identify the precise sources of information that affect search performance. This discussion leads to resolution of the seeming conflict between COI and the observation that some search algorithms perform well on a large class of problems.

[ IEEE | pdf ]

Comments

"References to “geographical structure[s],” “link structure[s],” search space “clustering,” and smooth surfaces conducive to “hill climbing” reinforce rather that refute the quasi-teleological conclusion that the success of evolutionary search depends solely on active information from prior knowledge [32]." Is this implying that the references are made references for the purpose of the search; or the references fit the search and therefor and specifically for the search?bandsci_{December 8, 2009
December
12
Dec
8
08
2009
06:57 PM
6
06
57
PM
PDT}

Is this implying that the references are made references for the purpose of the search; or the references fit the search and therefor and specifically for the search?
bandsci_{December 8, 2009
December
12
Dec
8
08
2009
06:56 PM
6
06
56
PM
PDT}

JT: And as far as the causes of mutation, I would have thought those were assumed to be uniformly distributed.
Though most studies indicate that genetic mutations are random with respect to fitness, the distribution of mutations is far from random. Not only are certain base substitutions more common than others, but some stretches of genome are more likely to experience mutations. However, for the purposes of a general discussion of evolutionary algorithms, purely random mutations can be assumed.
JT: Of course the natural selection component is not being considered here. But if that is is just attributes of the physical environment then that should be ultimately uniform as well.
The environment is hardly random. Energy and matter are not distributed uniformly through space. If you find a mote of water, there is a better chance there is another close by, and another, and another. Look at the ocean! Look at the sky! Because matter tends to clump, we have volumes with surfaces and objects with interactions. And then there's chemistry... The universe is structured, at the very least, by *proximity*. In terms of evolutionary search, there are hills to climb.Zachriel_{December 8, 2009
December
12
Dec
8
08
2009
06:53 PM
6
06
53
PM
PDT}

If the finite space we're talking about is non-uniform (the one leading to life, say), then there has to be a potential version of it that could be legitmately called uniform. That is the relevant one I think.JT_{December 8, 2009
December
12
Dec
8
08
2009
05:41 PM
5
05
41
PM
PDT}

Rob [35]:
Of course, in doing so, they fall into the trap that they describe as familiarity zones. When the problem definition specifies a certain finite space, it’s providing significant problem-specific information. A baseline random search takes advantage of that information, so it doesn’t actually follow the PrOIR as Marks and Dembski interpret it. Marks and Dembski say, “The ‘no prior knowledge’ cited in Bernoulli’s PrOIR is all or nothing: we have prior knowledge about the search or we don’t.” But something is always known about the problem, unless the problem is completely undefined.
It seems you could have knowledge to specify a finite space without it dictating a non-uniform distribution of that space. There is an implicit understanding of the universe in biology that allows for the conception of random mutations being a large part of the explanation for life. That might as well be the implicit framework for randomness assumed in the Dembski/Marks paper - the one already generally accepted. And as far as the causes of mutation, I would have thought those were assumed to be uniformly distributed. Of course the natural selection component is not being considered here. But if that is is just attributes of the physical environment then that should be ultimately uniform as well. Except supposedly the fact that the background radiation isn't smooth I guess means it wasn't uniform or something. And I just wanted to point out that Prof Ollofson seems to be rejecting PrOIR outright:
According to the PrOIR you must assume that the deck is well shuffled which clearly makes no sense.
JT_{December 8, 2009
December
12
Dec
8
08
2009
05:19 PM
5
05
19
PM
PDT}

JT[45], There are different levels of "infinite." On a countable set, probability is defined in the same way as on a finite set, by point probabilities. No problems there. For example, the numbers exp(-1)/n! give a probability distribution on the infinite countable set 0,1,2,... On an uncountable set, such as the interval [0,1], probability is defined through probability density functions which are integrated in order to find probabilities. As you point out, this means that each individual point has probability 0.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
02:56 PM
2
02
56
PM
PDT}

39- It seems that probability is not defined for an infinite range of values as if one value occurs then an event with probability 0 has occurred.JT_{December 8, 2009
December
12
Dec
8
08
2009
02:45 PM
2
02
45
PM
PDT}

P.Olofsson commented in 26: As for the math, frankly, the PrOIR is never used in probability. Consider D&M’s Formula (1). On the one hand they claim that it assumes the PrOIR due to “no prior knowledge.” But before Formula (1) they state that the deck is well shuffled which is a lot of prior knowledge and precisely the prior knowledge that warrants the uniform distribution. One must not confuse prior knowledge of the distribution of cards (which we do have) with prior knowledge of the location of the ace of spades (which we don’t have). As an example of the former, if I tell you I have a deck of card in my hand, you have no idea of the distribution of cards. It could be new and perfectly ordered, it could be well shuffled, it could be manipulated by me, etc. According to the PrOIR you must assume that the deck is well shuffled which clearly makes no sense. If Prof olofsson holds a deck of cards and I have no prior knowledge about the distribution any one guess would be the best one I can make so from my perspective the deck would be well shuffled no matter how it is actually ordered. In fact it could be said that from perspective of guessing something about a data random is the natural state of ANY data if there is no prior knowledge about it or how it was generated. So from this perspective I disagree with Prof.Olofsson and claim that well shuffled means no prior knowledge.Innerbling_{December 8, 2009
December
12
Dec
8
08
2009
02:11 PM
2
02
11
PM
PDT}

Mystic[43], I'll leave it to Bill to state what he is or is not. What does my quote "if we flip" have to do with your subsequent comment?Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
02:08 PM
2
02
08
PM
PDT}

Prof. Olofsson,
Congratulation Bill! So you’re a Bayesian now!
No, he's just not a frequentist. ;) If he were a Bayesian, he'd drag himself out of the PrOIR (yech!), and speak of exchangeability.
if we flip a fair coin repeatedly, the outcome of the 11th flip is independent of the first 10 flips
The "some to many" function g in Appendix A is just the converse of a function h -- reverse the arrows in Fig. 1. Drawing h uniformly and selecting x in Omega-hat to obtain solution omega = h(x) is equivalent to drawing omega uniformly from Omega. Why do things have to be so darned simple!?Mystic_{December 8, 2009
December
12
Dec
8
08
2009
12:38 PM
12
12
38
PM
PDT}

RoB[40]. Yes, I agree we should give credit where it is due. I am sure D&M have more papers in the making, and maybe these problems will be addressed.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
12:09 PM
12
12
09
PM
PDT}

Prof Olofsson @ 35, I agree with all of your points. To their credit, the Drs acknowledge some of the problems with the PrOIR, which they didn't do in previous papers. Unfortunately, they do very little to mitigate the problems, leaving us to question whether their PrOIR-based measures are non-arbitrary and useful.R0b_{December 8, 2009
December
12
Dec
8
08
2009
12:01 PM
12
12
01
PM
PDT}

Allen_McNeill said: “On the contrary, many (perhaps the majority) of the characteristics of living organisms are not adaptive.” What does that mean exactly?
This mean that even if we find some feature which is non-adaptive, we can still claim that it was the result of evolution, by calling it a spandrel. (An unintended by-product?) For example, if we cannot explain the mind's capacity for abstract mathematics in terms of adaptation, we can claim that the capacity for abstract mathematics is a by-product of some other feature that was adaptive. Isn't the science of evolutionary theory wonderful?Mung_{December 8, 2009
December
12
Dec
8
08
2009
11:40 AM
11
11
40
AM
PDT}

JT[36], There is no blunder. The example illustrates the difference in applying the PrOIR to the volumes and to the densities. If you apply it to the volume, it does not apply to the specific density (as you show) and if you apply it to the density it does not apply to the volume. The point is that both volume and density cannot be uniform on their respective ranges (assuming a continuous scale). There is no paradox in your example with the deck of cards since "between 1 and 1/26" means one of the 26 numbers 1, 1/2, 1/3,...,1/26. For finite spaces, the uniform distribution is preserved under transformation.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
11:32 AM
11
11
32
AM
PDT}

just forget that - I misread what Keynes was saying.JT_{December 8, 2009
December
12
Dec
8
08
2009
11:23 AM
11
11
23
AM
PDT}

This concerns the following objection to PrOIR from Keynes that Dembski mentions:
“Consider the specific volume of a given substance. Let us suppose that we know the specific volume to lie between 1 and 3, but that we have no information as to whereabouts in this interval its exact value is to be found. The Principle of Indifference [Bernoulli’s PrOIR] would allow us to assume that it is as likely to lie between 1 and 2 as between 2 and 3; for there is no reason for supposing that it lies in one interval rather than in the other. The specific density is the reciprocal of the specific volume, so that if the later is v the former if 1/v . Our data remaining as before, we know that the specific density must lie between 1 and 1/3 , and, by the use of the Principle ... as before, that it is as likely to lie between 1 and 2/3 as between 2/3 and 1/3 .”
There seems to be a blunder here because the observation should have been that the specific density is as likely to be between 1 and 1/2 as it is to be between 1/2 and 1/3 (so, 1,1/2,1/3 is the sequence; not 1,2/3,1/3 as erroneously stated in the above.) But also, such a probability distribution would be contingent on prior knowledge about the distribution of its reciprocal. Without that knowledge, presumably we could conclude that it was as likely for the specific density to be between 1 and 1/6 as between 1/6 and 1/3. So the relevant issue above is not infinite alternatives as subsequently speculated:
Bertrand [3] was “so much impressed by the contradictions of geometrical probability that he wishes to exclude all examples in which the number of alternatives is infinite” [44].
Even with a deck of cards you could say the sequence number of the card selected at random is as likely to be between 1 and 26 as between 26 and 52, and therefore the reciprocal of the sequence number of the selected card is as likely to be between 1 and 1/26 as between 1/26 and 1/52. (Maybe the problem lies with values less than 1, not infinite alternatives.)JT_{December 8, 2009
December
12
Dec
8
08
2009
11:05 AM
11
11
05
AM
PDT}

ROb[34], Yes, in [30] I was not addressing the paper but made a general comment on [28]. They still have the same problem as in the juvenile delinquency example (where they claim a uniform distribution is not warranted). If you have a uniform distribution over the deck, you don't get a uniform distribution over the dichotomy ace/no ace and vice versa. Anyway, my main point is that Bernoulli's PrOIR is never used in applications. We know more now than we did 300 years ago.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
10:37 AM
10
10
37
AM
PDT}

Prof Olofsson and Heinrich, in fairness to Drs Marks and Dembski, they address the problem of transformations in section III, and they get around it by restricting themselves to finite discrete spaces. Of course, in doing so, they fall into the trap that they describe as familiarity zones. When the problem definition specifies a certain finite space, it's providing significant problem-specific information. A baseline random search takes advantage of that information, so it doesn't actually follow the PrOIR as Marks and Dembski interpret it. Marks and Dembski say, "The 'no prior knowledge' cited in Bernoulli’s PrOIR is all or nothing: we have prior knowledge about the search or we don’t." But something is always known about the problem, unless the problem is completely undefined.R0b_{December 8, 2009
December
12
Dec
8
08
2009
10:23 AM
10
10
23
AM
PDT}

I understood! You're right, of course. There's some work (which you may know better than me) on extending the Jeffrey's priors to multivariate distributions, but the last I heard there was still work to be done.Heinrich_{December 8, 2009
December
12
Dec
8
08
2009
09:24 AM
9
09
24
AM
PDT}

Nope. :(Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
09:18 AM
9
09
18
AM
PDT}

Post [30], should be "p squared." Trying again: p2Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
09:18 AM
9
09
18
AM
PDT}

Heinrich[28], How could it be solved? For example, if p is uniform on [0,1], then p2 is not uniform and vice versa. There's no way out. Then again, Bayesian inference is all about the posterior distribution. You never assume a uniform prior and then treat it as the truth. Gotta have data.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
09:10 AM
9
09
10
AM
PDT}

Allen_McNeill said: "On the contrary, many (perhaps the majority) of the characteristics of living organisms are not adaptive." What does that mean exactly?Collin_{December 8, 2009
December
12
Dec
8
08
2009
09:01 AM
9
09
01
AM
PDT}

Bill has previously argued hard for this paradigm against the subjective probability interpretation of Bayesianism, so I assume he has changed his views, which is, of course, perfectly fine.
IIRC, Dr. Dembski explicitly used a Bayesian approach to analyse the Jesus' tomb claim. One well-known problem with PrOIR is that it only works on specific scales - a transformation makes the distribution non-uniform. If Dr. Dembski could solve that, it would be a substantial contribution, and would revitalise objective Bayesianism.Heinrich_{December 8, 2009
December
12
Dec
8
08
2009
08:50 AM
8
08
50
AM
PDT}

About footnote [7] in the D&M paper. Seriously, to probabilists such as Bill and myself, the original NFL Theorem by Wolpert and MacReady is completely trivial, essentially stating that if we flip a fair coin repeatedly, the outcome of the 11th flip is independent of the first 10 flips (see Haggstrom's proof). Now, W&M did not realize the simple proof of their result, but a triviality it is nevertheless. In contrast, Godel's Incompleteness Theorem is one of the most profound mathematical results, killing the hope that mathematical truth and provability are equivalent. Anybody who has read its proof knows that it invokes an ingenious idea (Godel numbers) and is far from trivial. Leave poor old Kurt alone!Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
08:49 AM
8
08
49
AM
PDT}

Regarding my post [17], as D&M point out, in the frequentist paradigm Bernoulli's PrOIR has no meaning. Bill has previously argued hard for this paradigm against the subjective probability interpretation of Bayesianism, so I assume he has changed his views, which is, of course, perfectly fine. There isn't much of a fight between the two camps these days. As for the math, frankly, the PrOIR is never used in probability. Consider D&M's Formula (1). On the one hand they claim that it assumes the PrOIR due to "no prior knowledge." But before Formula (1) they state that the deck is well shuffled which is a lot of prior knowledge and precisely the prior knowledge that warrants the uniform distribution. One must not confuse prior knowledge of the distribution of cards (which we do have) with prior knowledge of the location of the ace of spades (which we don't have). As an example of the former, if I tell you I have a deck of card in my hand, you have no idea of the distribution of cards. It could be new and perfectly ordered, it could be well shuffled, it could be manipulated by me, etc. According to the PrOIR you must assume that the deck is well shuffled which clearly makes no sense. The only possible practical use of the PrOIR would be as a prior in a Bayesian setting, but then it is only there to be replaced by the posterior after data has been gathered. In other words, we can only invoke the PrOIR if we intend to abandon it. As D&M do not device any mechanism for updating to posterior distributions, their results have no practical applications.Prof_P.Olofsson_{December 8, 2009
December
12
Dec
8
08
2009
08:36 AM
8
08
36
AM
PDT}

#24 Heinrich I think the phrase you quote is a bit imprecise. The key phrase is: "all some-to-many mappings of a uniform search space result in a new search space where the chance of doing better than p is 50-50". (which is explained in more detail in the paper on page 2649 and appendix A). As I understand it this means that if you select a subset of all blogs on the internet, without knowing anything about how blogs are distributed on the internet, then you have as much chance of decreasing your probability of getting a blog authored by WAD as you have of increasing the probability. This is fairly obviously true. I just think it is irrelevant to evolution.Mark Frank_{December 8, 2009
December
12
Dec
8
08
2009
04:48 AM
4
04
48
AM
PDT}

The assumption is conserved under mapping. If the probability of finding a target in a search space is p, then the problem of finding the target in any subset of the search space is p.
Isn't this just wrong? It's like saying that the probability of finding a Dr. Dembski authored a blog post on the internet is equal to the probability of that he authored one on UD. It's apparent from the paper that what is meant is that the assumption of insufficient reason is conserved over a summation of all possible mappings.Heinrich_{December 8, 2009
December
12
Dec
8
08
2009
02:02 AM
2
02
02
AM
PDT}

I still can't get my head round treating the evolutionary algorithm as a discrete finite random variable. First it is necessary to define more precisely what is actually meant by the evolutionary algorithm. To make any sense at all then the algorithm has to include the mechanism(s) of variation, of selection, and the environment in which they operate - these all affect the probability of the outcome and therefore in ID terms provide "exogenous information". Given this: 1) It is not discrete - conceivable algorithms vary continuously. 2) It is not finite. There are an infinite number of conceivable algorithms. 3) Most importantly - it is not a variable. The mechanisms and environment were not selected from all conceivable such mechanisms and environments. They are a given. I think the source of confusion is illustrated by the reference to Conway Morris's observation on convergence. It may well be that given the evolutionary algorithm that the result is to some extent determined. That does not imply that the evolutionary algorithm was selected among many to provide that outcome. The algorithm existed first. The outcome followed. It is like saying what are the chances of the laws of gravity being such that the moon orbits the earth.Mark Frank_{December 8, 2009
December
12
Dec
8
08
2009
01:14 AM
1
01
14
AM
PDT}

Prev 1 2 3 Next

You must be logged in to post a comment.

Leave a Reply