Uncommon Descent Serving The Intelligent Design Community

Claude Shannon: the man who failed to transform our understanding of information

Share
Flipboard
Print
Email

Well, Columbia’s Rob Goodman thinks he did, at Aeon:

Shannon’s ‘mathematical theory’ sets out two big ideas. The first is that information is probabilistic. We should begin by grasping that information is a measure of the uncertainty we overcome, Shannon said – which we might also call surprise. What determines this uncertainty is not just the size of the symbol vocabulary, as Nyquist and Hartley thought. It’s also about the odds that any given symbol will be chosen. Take the example of a coin-toss, the simplest thing Shannon could come up with as a ‘source’ of information. A fair coin carries two choices with equal odds; we could say that such a coin, or any ‘device with two stable positions’, stores one binary digit of information. Or, using an abbreviation suggested by one of Shannon’s co-workers, we could say that it stores one bit.

But the crucial step came next. Shannon pointed out that most of our messages are not like fair coins. They are like weighted coins. A biased coin carries less than one bit of information, because the result of any flip is less surprising. Shannon illustrated the point with this graph. You see that the amount of information conveyed by our coin flip (on the y-axis) reaches its apex when the odds are 50-50, represented as 0.5 on the x-axis; but as the outcome grows more predictable in either direction depending on the size of the bias, the information carried by the coin steadily declines.

This is where language enters the picture as a key conceptual tool. Language is a perfect illustration of this rich interplay between predictability and surprise. We communicate with one another by making ourselves predictable, within certain limits. Put another way, the difference between random nonsense and a recognisable language is the presence of rules that reduce surprise.More.

The unfortunate reality is that the vast majority of science writers, while working within an information paradigm even as they write, write as though the Big Fix (the universe as simply matter and energy) is still just around the corner, along with the multiverse and the success of string theory. And the sure-thing tale of human evolution.

Introduction to Evolutionary Informatics

and

New Scientist: Human evolution “more baffling than we thought” (after all this time)

The data processing inequality (DPI), derived from Shannon information, shows any undirected process cannot increase mutual information I(X;Y) between a target (X) and an organism (Y). I(X;Y) = H(X) - H(X|Y) If we say the goal is increasing mutual information between an organism and a complex feature such as an eye, and that evolution is not directed towards producing eyes, then evolution cannot account for complex functionality such as eyes. Say Y is the current generation and Z is the next generation of an organism. The DPI states that if X does not tell us anything about generating Z from Y (i.e. evolution is not directed towards X), H(Z|Y) = H(Z|Y,X), then the mutual information between organism and X can never increase during the course of evolution, I(X;Y) >= I(X;Z). So, evolution, insofar as it is undirected towards complex functionality, cannot account for its occurrence. The information that allows complex functionality, such as eyes, to form must come from outside of evolution. This is really what Dembski's CSI metric and conservation of information boil down to. CSI is the mutual information between the event and the target, and conservation of information says that without impartation of external information (exogenous) about the target, then CSI cannot be increased. So as you can see, Dembski's work is a reformulation of the data processing inequality.EricMH
September 2, 2017
September
09
Sep
2
02
2017
11:02 AM
11
11
02
AM
PDT
August 31, 2017
August
08
Aug
31
31
2017
04:42 PM
4
04
42
PM
PDT
Shannon is the man who fails to transform your understanding of information if you don't have the math background and motivation to read his published papers.groovamos
August 31, 2017
August
08
Aug
31
31
2017
12:49 PM
12
12
49
PM
PDT
The thing you must understand is that Mr. Shannon was originally talking about "signal" and "noise" in WW2 radar signals. Later, he worked for AT&T on digital telephone signals. Reliably sorting Signal from Noise in a VERY narrow context is only distantly related to things like human speech. But the fact that enhancing the quality of radar reception turned out to have direct application to making general statements about human speech and writing says a lot about how humans (who of course created RADAR and digital telephones) create Information generally. That is, having already created written languages, it was simple for humans to create machines that transmit, receive, and display information of interest to humans. Sorting signal from noise is of course a thing that human children learn as they build their vocabulary. Processing signal into information in context (is the wind "blue"? or did the wind "blew"?) requires a fair piece of processing.vmahuna
August 31, 2017
August
08
Aug
31
31
2017
12:41 PM
12
12
41
PM
PDT