Eric Holloway: How can we measure meaningful information?

_{Denyse O'Leary

December 27, 2018

Information, Intelligent Design, specified complexity}

Share: Facebook; Twitter; LinkedIn; Flipboard; Print; Email

Neither randomness nor order alone create meaning. So how can we identify communications in a scientifically meaningful way?

Dropping a handful of toothpicks on the table seems to produce a different sort of pattern than spelling out a word with toothpicks. Surprisingly, this intuitive distinction is harder to make in math and the sciences. Algorithmic specified complexity (ASC) enables us to distinguish them.

Neither Shannon information nor Kolmogorov complexity work well for this purpose.

This leads us to a third concept, algorithmic specified complexity (ASC). ASC solves the problem by combining the two measures. ASC states that an event has a high amount of information if it has both low probability and a concise description. This matches our intuition much better. More.

Eric Holloway has a Ph.D. in Electrical & Computer Engineering from Baylor University. He is a current Captain in the United States Air Force where he served in the US and Afghanistan He is the co-editor of the book Naturalism and Its Alternatives in Scientific Methodologies. Dr. Holloway is an Associate Fellow of the Walter Bradley Center for Natural and Artificial Intelligence.

Also by Eric Holloway: Human intelligence as a halting oracle

and

Does information theory support design in nature?

Comments

@Bob O'H, I don't have a ready answer for your question regarding how we detect meaning, though it is an interesting one. Best I can determine is the mind must be a halting oracle in order to create and detect meaning. But, long before ASC was discovered, we were using our meaning detection ability to learn about reality and make predictions. The subjectivity is inescapable, but not necessarily a problem. For instance, with compression, though we cannot calculate Kolmogorov complexity itself, we know that whatever compression we come up with is an upper bound on the true complexity. Thus, we know the complexity cannot be higher. In a similar fashion, OASC is a lower bound on ASC, so we know that positive OASC indicates positive ASC. This means the subjective aspect of OASC does not present a problem insofar as design detection is concerned.EricMH_{January 2, 2019
January
01
Jan
2
02
2019
11:54 AM
11
11
54
AM
PDT}

Bob O'H:
So if ASC existed (in some sense) before it was invented, the question still remains: how did we detect meaning before ASC was invented? You haven’t answered that question.
The question doesn't make any sense, Bob. If it existed then it wasn't invented.
gravity works without any mind or intelligence.
How do you know that gravity would exist without any mind or intelligence? ID says gravity works because it was designed by a mind/ intelligence.ET_{January 1, 2019
January
01
Jan
1
01
2019
10:00 AM
10
10
00
AM
PDT}

EricMH - So if ASC existed (in some sense) before it was invented, the question still remains: how did we detect meaning before ASC was invented? You haven't answered that question. There is a difference between detecting meaning and gravity: gravity works without any mind or intelligence, but it's impossible to detect meaning without using intelligence.
The subjective aspects are bounds on the objective measurement.
Indeed, but I don't see how you can get around them - they'll still be there.
... The chance hypothesis probability we can calculate exactly given a mathematical theory. Selecting a chance hypothesis can also be objective given a background theory.
And the choice of background theory will have subjective elements: you will have to make assumptions to make it tractable, and these will involve subjective decisions.
For example, if our background theory is the universe is computable, then the chance hypothesis self information is lower bounded by the algorithmic information of the event.
The bounds on ASC are not the same as ASC itself, and might not be terribly useful. The "tornado in a junkyard" theory that is often used (i.e. every item in a string is iid) is a bound, but is fairly useless, as decades of work on evolutionary computation have show that it massively under-estimates the probabilities of getting a pattern which has a higher fitness.Bob O'H_{January 1, 2019
January
01
Jan
1
01
2019
03:29 AM
3
03
29
AM
PDT}

@Bob O'H, I'd say ASC was discovered like the law of gravity. We weren't suspended in mid air before Newton discovered the law of gravity, and the rest of modern physics. Likewise, a recent discovery of a way to measure meaning does not entail no one could understand anything before ASC. The subjective aspects are bounds on the objective measurement. For example, any compression algorithm you pick will be an upper bound on Kolmogorov complexity. The chance hypothesis probability we can calculate exactly given a mathematical theory. Selecting a chance hypothesis can also be objective given a background theory. For example, if our background theory is the universe is computable, then the chance hypothesis self information is lower bounded by the algorithmic information of the event. The context is the one element of ASC I'm not sure about. One possible objective way to choose the context is the Kolmogorov minimal sufficient statistic (KMSS). In which case, if the self information is also the algorithmic information, then ASC becomes identical to the KMSS. Thus, the KMSS seems to be in some way the optimal ASC of an event if the universe is computable. Consequently, if high KMSS events occur with regularity in the universe, then some uncomputable force is at work in the universe.EricMH_{December 31, 2018
December
12
Dec
31
31
2018
10:43 AM
10
10
43
AM
PDT}

EricMH -
However, ASC does appear to be a necessary component of detecting meaning.
Really? So how did we detect meaning before it was invented? Or are you claiming that has it only been in the last 3 years that mankind has been able to detect meaning?
Regarding ASC’s objectivity, Kolmogorov complexity provides an observer independent way to measure ASC for a given item.
Yes, once you have the specification, and the model from which to calculate the probability. But both of these will (in practice) require subjective decisions to elicit. Unless you can show some mind-independent way of specifying them.Bob O'H_{December 31, 2018
December
12
Dec
31
31
2018
08:08 AM
8
08
08
AM
PDT}

@Bob O'H, I would not say that ASC is a sufficient measure of meaning. Granting that I am an intelligent (!?) agent, I can draw a straight line on a piece of paper. Given there are a wide variety of lines that I could have generated, and the straight line is highly compressible, the straight line would have high ASC, but wouldn't necessarily mean anything. However, ASC does appear to be a necessary component of detecting meaning. Regarding ASC's objectivity, Kolmogorov complexity provides an observer independent way to measure ASC for a given item. And, as GP puts it more simply, given an independent specification we can partition a collection, and can eliminate chance if the partition is small enough.EricMH_{December 31, 2018
December
12
Dec
31
31
2018
04:33 AM
4
04
33
AM
PDT}

gpuccio @2: “Meaning is never intrinsic. It is always designed. Design outputs meaning and function to matter in the form of recognizable configurations, but meaning and function come from consciousness and the mind.” Agree. Even the exact meaning and the phonetic sound of this combination of characters in english: “read” could vary depending on the time context, which could be hinted by surrounding cues: I read gpuccio’s excellent comment earlier today. Peter might want to read it too someday. PavelU will read it but won’t understand it. GP’s comments are usually read by many curious onlookers.jawa_{December 30, 2018
December
12
Dec
30
30
2018
02:05 AM
2
02
05
AM
PDT}

gpuccio has it right- the information (meaning) flows from artisan to artifact. Also, meaning, like functionality, is an observation. To measure the meaning would we have to determine the context and then apply the correct definitions to the words used?ET_{December 29, 2018
December
12
Dec
29
29
2018
09:09 AM
9
09
09
AM
PDT}

Bob O'H (and Eric Holloway): Of course meaning is not intrinsic in the object: it is inpoutted into the object by the designer as a specific configuration, and recognized by the observer. And of course you are right: meaning (or function) are "inserted into the specification": indeed, they are the specification. So, again, the specification is the basic idea. But the complexity linked to the specification is important too, because it allows us to avoid those simple cases of pseudo-specification that can easily arise as contingent results, because they are not complex, and a realistic non design system can certainly generate them by chance. So, the two components are the specification and the complexity, as Dembski has always said just from the beginning. Specified complexity allows us to infer design. Now, specification is probably the point that generates some confusion. It can be defined in different ways. As you probably know, I usually stick to my definition of functional specification, because it can be made very objective and explicit, and it can be perfectly applied to biological contexts. However, functional specification is certainly a subset of a wider concept of specification. That's what Eric probably means by ASC: a general definition of specification. Maybe too general: "a concise description" is, indeed, a too concise description. But I am certain that Eric can provide more details, so that the concept does not remain vague. Intuitively, I think that this is a new formulation of Dembski's ideas in his famous paper about specification. I have always had some difficulties with that paper, but maybe it's only my scarce understanding of mathematical concepts. I think that George Montañez, too, has recently tried to give a general foundation to the concept of specification. Seems interesting, but again a little too mathematical for me. My simple idea is that any objective definition that can generate a binary partition in the search space can be used as specification. A nd if the target space defined by that partition is really, really small (IOWs, if the specified information is more than 500 bits), and yet a result in that target space is really observed, then we can safely infer design. The important points are: a) Any specification can be used, provided that it respects the rules given here. b) The specification must be explicit and objective. IOWs, once defined, it must be possible for anyone to apply it to objects. This is an important point: even if a word has different menaings in different contexts, our specification must objectively define the meaning and the context to be recognized. So, if the meaning we recognize implies that "gift" means "present", the general context must be an English phrase. So, while the recognition of the specification is made by some observer, it must be objectively defined, and it must be base on objective references. c) We cannot use the contingent configuration of the observed result to define the specification. This is very important. It simply means, if we have a string of digits like 3.14159265359, we cannot use a specification like: "a string of digits like 3.14159265359". That is cheating: we are using the observed contingency to build our function or menaing. But we can certainly say: "a string that correctly represents the first 12 decimal digits of pi", because now the specification is objective, based on an external and pre-existing reference, and does not used the observed contingency. In the same way, we cannot observe a string of random "words", and then create a language where those words have good meaning. The language must already exist before the result is observed. We cannot use the contingency in the observed result to create the specification. In the same way, we cannot take a random string of digits and then set it as the key to a safe. Again, we are creating the specification using the contingency we observe, we are not just recognizing a specification based on some external and independent reference. Again, this point is very important and must be correctly understood. There is nothing "subjective" in a correct specification. d) Finally, a specification can be based on order, but in that case we must be sure to exclude necessity as a cause of what we observe. IOWs, when order is implied, we must always reason in terms of Kolmogorov complexity. Those simple few rules work very well with my definition of functional specification. I think that they will probably work with any more general definition of specification. provided it is objective and explicit and detailed. I hope that Eric can find the time to comment on these points, and maybe give us more details about the concept of ASC :) .gpuccio_{December 29, 2018
December
12
Dec
29
29
2018
08:53 AM
8
08
53
AM
PDT}

To a Brit is means a present, but to a German it means poison.
In Danish and Norwegian 'gift' means both 'poison' and 'married'.
So, I can’t see how your discussion about GIFT or other language configurations has any relevance: meaning is inherent in specific configurations that use existing languages and conventions to express well understandable information.
I think the point is that meaning (as you later state) isn't intrinsic: it is inferred by the observer. So meaning has to have an external context, and different contexts can lead to different meanings (or, indeed, none at all). I assume the meaning that ASC is mean to measure is inserted into the specification, but that is barely alluded to in the article.Bob O'H_{December 29, 2018
December
12
Dec
29
29
2018
01:51 AM
1
01
51
AM
PDT}

DaRook: The point is simple enough. If you have the following phrase in English: "I consider the argument at comment number 1 of this thread as a gift, because it allows me to make an important point about meaningful information" that phrase hase high specified information. Because it has a definite meaning in an existing language, and anyone who knows English language can understand that meaning. It is certainly true that an Australian Aborigine (if he does not know Engleish), or even an italian doctor who does not know that language (there are many of them, believe me), would not understand. And so? If you have the first 1 million decimal digits of pi. they will not mean anything to an Australian Aborigine. Probably. Indeed, I have no reasons to doubt mathenatical skills of Australian Aborigines. So, let's say they will mean nothing to some italian art historians, who may not be fans of mathematics. Again, and so? I know nobody who would deny that some written form of the first 1 million digits of pi must be designed. Even enemies of ID, when confronted with an example of that kind of information found on some rocky wall on a distant planet of which we know nothing would warrant a design inference. So, I can't see how your discussion about GIFT or other language configurations has any relevance: meaning is inherent in specific configurations that use existing languages and conventions to express well understandable information. Including the digits of pi. So, "gift" will mean a present in an English context, and poison in a german context. And so? If I know both languages (I don't) I will be able to understand the right meaning in each context. Meaning is never intrinsic. It is always designed. Design outputs meaning and function to matter in the form of recognizable configurations, but meaning and function come from consciousness and the mind.gpuccio_{December 28, 2018
December
12
Dec
28
28
2018
04:44 PM
4
04
44
PM
PDT}

Information rides on increased entropy. High entropy has a higher carrying capacity than low entropy. It is not information itself. Spelling a word with toothpicks only seems less probable because we put information on that pattern of toothpicks. To an Australian Aborigine, it is as meaningless as any other pattern of toothpicks. To an Arab, the letter combination GIFT holds no meaning. To a Brit is means a present, but to a German it means poison. GIFT holds no intrinsic meaning in and of itself.DaRook_{December 28, 2018
December
12
Dec
28
28
2018
03:36 PM
3
03
36
PM
PDT}

You must be logged in to post a comment.

Leave a Reply