Uncommon Descent Serving The Intelligent Design Community

Biologist Wayne Rossiter on Joshua Swamidass’ claim that entropy = information

Share
Facebook
Twitter
LinkedIn
Flipboard
Print
Email

From Wayne Rossiter, author of Shadow of Oz: Theistic Evolution and the Absent God, at Shadow of Oz:

Swamidass surprises us with a very counter-intuitive statement:

“Did you know that information = entropy? This means the 2nd law of thermodynamics guarantees that information content will increase with time, unless we do something to stop it.”

Of course, he is arguing much more than just the observation that entropy = information. He’s arguing that the 2nd Law of Thermodynamics guarantees an increase in information. This seems very sloppy to me. Yes, if we consider time as a variable, then the universe will develop pockets of increased information (complexity?), while still experiencing a global loss of information and complexity. The universe putatively began as a maximally dense, maximally hot locus. As it expands, heat and potential energy will dissipate, even if locally, structure and information emerges.

In some ways, Swamidass’s argument sounds similar to that of Lawrence Krauss (a devout nonbeliever). Swamidass is basically arguing that the universe begins at maximum energy, but minimum complexity (no information-rich structures, just the simplest atoms–hydrogen and helium). It then moves through stages of more complex (information-rich) structures, even as it loses available energy. It then ends largely as it began, and those complex structures reduce and decay back to a uniformly distributed universe, devoid of free energy, and the universe becomes a “sparse soup of particles and radiation.”It emerges as simplicity and ends as simplicity.More.

If Swamidass (or any similar person is funded by Templeton), he can argue anything he wants, including any form of fake physics. Making sense is optional.

Sorry, gotta go fill up another trash bag full of suddenly appearing Boltzmann brains … Quite the plague this year.

See also: 2016 worst year ever for “fake physics”?

Follow UD News at Twitter!

Comments
Thanks, guppio. No problem. Looks like you've got an awesome OP up, and I look forward to checking it out. You don't have to agree with me on all the entropy details. Very good exchange anyway and it has been useful. In any event, we certainly agree on the larger implications. I'm just hoping something I wrote may pique your interest or cause any who may stop by and read this exchange to sit up and say, "Wait a minute, maybe I need to think through this again in more detail . . ." That is all I can hope for. :)Eric Anderson
March 24, 2017
March
03
Mar
24
24
2017
09:11 AM
9
09
11
AM
PDT
Eric Anderson: Thank you for the further comments. Sorry if I am late in answering, but I have been busy with a new OP in the last day. I don't know what to say. I still think the the same concepts that are valid for informational entropy in strings do apply to thermodynamic entropy in physical systems, but as I have said I am not a physicist and therefore I cannot really explain the correspondence. Maybe KF could help here. I see that you are not comfortable with this idea, but perhaps we should deepen our understanding of thermodynamics and of the probabilistic approach to the second law. However, for informational entropy I think we have reached a very good understanding. It's always a pleasure to have a good exchange with you. :)gpuccio
March 23, 2017
March
03
Mar
23
23
2017
03:59 PM
3
03
59
PM
PDT
gpuccio @50: Returning to the other part of your comment:
The point is: most random configurations, both for strings and for physical systems, are different . . .
Yes, random configurations are different, by virtue of their randomness. But what is it that we think we are trying to describe in a physical system?
Ordered states are compressible, and usually have a vey specific macro-behaviour. But their entropy is low. Moreover, they are exceedingly rare.
Ordered states, in the sense of functional organization are exceedingly rare. But simple order is not so rare. Crystals, waves, snowflakes -- many things that are driven by law-like processes tend toward mere "order" (and, therefore, compressibility in their description).
Random states . . . are all different, and scarcely compressible. Therefore, they have high entropy. But their macro-behaviour is strikingly similar (the “homogeneity” of which you speak).
And thus we return to one of the key questions, but I like the words you've used, so let me reframe it: Why do we think we get to describe an ordered state in macro-behavior terms, but we have to describe a random state in micro-behavior terms? I could just as well decide I want to do the opposite. In which case my brief description of the random system at the macro level would be highly compressible, while my lengthy and detailed description of the ordered (at least a functionally organized) system at a micro level would not. The point is, we are making a decision about what to describe. This brings me back to a fundamental point. There is no information in physical things by their mere existence. Even if we were to grant the (somewhat questionable, it seems to me) idea that entropy drives toward random states and, further that for some reason we have to describe random states in micro terms -- even if we grant all that, it doesn't mean there is any information contained in the system. All it means is that it requires more information for us to describe the system, to whatever level of detail we have decided is sufficient for our purposes.Eric Anderson
March 23, 2017
March
03
Mar
23
23
2017
10:25 AM
10
10
25
AM
PDT
gpuccio: Thanks for your comments. I am fully on board with your ultimate conclusion:
Swamidass is talking of entropy, not of information. But he is trying to suggest that entropy means information. And that is not true.
And also agree with the general thrust of how you view things. I appreciate you sticking with me this long to delve into the nuances! --- Just one quick observation this evening, and then I'll hopefully have a chance to respond with more detail tomorrow. The problem with strings is that we often view an entropic string (random string) as being made up of a sequence of various characters. However, in these kinds of discussions everyone almost always ignores how or what made the changes to the string. So in a typical example we start with a sense string (say, made up from the 26 characters of the English alphabet) and then introduce (who introduces? what introduces?) random characters to the string. Sure enough, the string ends up less ordered and less compressible than when we started. Entropy in the real world doesn't act that way, it seems. Entropy doesn't randomly drive particles toward one of 26 possible states. It isn't "randomly" picking states and assigning them to particles, the way our random characters get jumbled in a string. If we are going to talk about strings, a more apt analogy to the physical world would be if we were to start with a sense string and then introduce a bunch of "A" characters over time. Nothing but "A". Over and over. Always driving toward the same simple state. Or, if we prefer a digital example, let's say we have our files stored on a hard drive that, unfortunately, over time replaces bits with a zero. We sometimes think of entropy as being random. But it is "random" only in the sense that it is stochastic and may or may not impact a particular particle at a given instant. But it is not at all "random" in the sense of where it is driving. It isn't "randomly" deciding whether to drive toward a bunch of different energy states (like our random character intrusions into a string). As a lawful process, it always drives in the same direction. So it seems to me there is a basic disconnect between the idea of how we get a random string and how entropy drives toward a particular state in the physical world. We could no doubt come up with some real-world examples in which entropy would -- temporarily as it moves away from complex functional organization -- result in a more heterogeneous state. But ultimately it seems it is inexorably driving in a single direction. It drives toward homogeneity, toward sameness, toward blandness, toward the "heat death of the universe", whatever we want to call it. But ultimately it is driving downhill -- always in at least the same general direction.Eric Anderson
March 22, 2017
March
03
Mar
22
22
2017
06:38 PM
6
06
38
PM
PDT
Eric Anderson: Well, I am not a physicist, and I usually avoid to deal with problems regarding the second law. I can only say that, intuitively, I think that in essence there is no difference between the concept of Shannon entropy for strings and the concept of entropy in physical systems. I believe that such a think has been demonstrated, too, but probably KF could say something more specific about that. His quote at #15 can certainly be of help. I will try to explain how I see things, very simply. You say: "Now, I ask: Q: In the three-dimensional, physical world, does entropy generally drive toward homogeneity/uniformity or toward heterogeneity? A: It drives toward homogeneity/uniformity." I am not sure that is correct. To understand what I mean, let's start with strings. Let's imagine a long string, say 600 characters. Each character is a value of a three letter alphabet, R, G and B. Therefore, the potential maximum information in such a string is 3^600, IOWs about 951 bits. Now, let's imagine that the string is in reality a sequence of pixels on a monitor, and the three letters are really the basic colors in the RGB system. That carries us immediately from the realm of pure information to the realm of physical events. Now, let's start with a highly ordered sequence: 200 R, 200 G, 200 B. In an informational sense, we see a line made of three parts, each of a different color. That's a very ordered state. From our discussion about strings, we know that it is also highly compressible. Moreover, it is exceedingly rare: only one sequence corresponds to such an ordered state, out of the 3^900 possible sequences. Now, let's say that instead we start from a random sequence of the three letters. What will we see? Most probably, a line in some shade of grey. Not white, because white is a very ordered state too: the most similar thing to pure white, I believe, should be a regular sequence of RGBRGBRGB and so on. Now, let's say we start from a completely different random sequence. It is completely different from the first random one, and if we want to specify it instead of the first random one, we really need to give all the specific values of the sequence. IOWs, both the first random sequence and the second one are different and scarcely compressible, like most random sequences. So, what do we see? With great probability, a very similar line in a similar shade of grey. The point is: most random configurations, both for strings and for physical systems, are different, scarcely compressible and therefore have high entropy, be it Shannon entropy or thermodynamic entropy. Ordered states are compressible, and usually have a vey specific macro-behaviour. But their entropy is low. Moreover, they are exceedingly rare. Random states, instead, constitute the bulk of configurations. Those configurations are all different, and scarcely compressible. Therefore, they have high entropy. But their macro-behaviour is strikingly similar (the "homogeneity" of which you speak). That's how I see the problem. But, again, I am not an expert of the second law and its implications. The single important point is: entropy, be it Shannon's or thermodynamic, has nothing to do with information as we use the word in common language. The meaning of the word "information" in common language corresponds perfectly to the more rigorous concept of "specified information" or "functional information". Swamidass is talking of entropy, not of information. But he is trying to suggest that entropy means information. And that is not true.gpuccio
March 22, 2017
March
03
Mar
22
22
2017
11:50 AM
11
11
50
AM
PDT
gpuccio: There are a few more detailed nuances we can discuss, but I wanted to circle back to a basic starting point: We've established that a more repetitive (more homogeneous, less heterogeneous) string is, in principle and under basic algorithms, more compressible. Now, I ask: Q: In the three-dimensional, physical world, does entropy generally drive toward homogeneity/uniformity or toward heterogeneity? A: It drives toward homogeneity/uniformity. What, then, is the basis for the claim that entropy produces more information content? ----- Even setting aside Dr. Swamidass' complete failure of understanding in referring to "information = entropy", what is it that makes someone think entropy will drive toward a less compressible description (given that he also seems stuck in the Shannon bit-calculation mistake, as far as biological information is concerned)?Eric Anderson
March 21, 2017
March
03
Mar
21
21
2017
10:28 AM
10
10
28
AM
PDT
Eric Anderson: Just a comment, for the moment. If the system is set in a very general way, we can certainly think of it as receiving and executing automatically both messages and algorithms. That's what I meant when I made a reference to a Turing machine. Think of the receiver as a universal Turing machine that can automatically receive strings, and treat them as pure outcomes or as computational algorithms that generate the outcome. A universal Turing machine is a very useful abstraction when we reason about information problems. So, in this case, transmitting the algorithm to compute pi digits (or any other outcome that is computable) will be in principle the same thing as transmitting an outcome as it is. In both cases, we send a series of bits. The algorithm is just a series of bits that, when received by the universal Turing machine, performs a computation and then generates the outcome and stops. So, in this context, transmitting the compression algorithm is not different from transmitting an outcome. No special understanding is required on the part of the receiver, which works automatically, except for the initial setting of the communication protocol and of the universal Turing machine, which is implemented so that it can receive and execute any kind of algorithm. I say this because, if we want to reason in terms of Shannon entropy and Kolmogorov complexity, we must set the system in a certain way, so that our results are really independent from any added knowledge, except for the initial setting. I am looking forward to your further considerations!gpuccio
March 20, 2017
March
03
Mar
20
20
2017
11:09 AM
11
11
09
AM
PDT
Origenes: I agree with what you say. I think we are saying similar things, it's really a question of words and definitions. Maybe I can explain it better this way: You speak of function as indicating the conscious purpose of the designer. That is perfectly fine for me. I have always stressed the role of consciousness in design: design derives from a conscious representation, and that conscious representation implies both cognition (the understanding) and purpose (a feeling connected to the meaning of the desired outcome). I have also always said that conscious experiences cannot ever be defined in an outward, purely objective way. "Purpose" cannot be defined as an objective concept, because it arises in feeling. Even "meaning" has no objective definition, independent from the conscious intuition of a subject. That said, you will probably understand better what I mean with "function" as the basis for design inference if you think of it not as "purpose", but rather as a "functionality" in the object. The functionality is a specific configuration which has been given to the object by the designer, in a context which depended on his conscious view of the whole and on his purpose. However, the specific configuration which makes the object functional so that it can do what the designer wanted to get from that specific object, well, that configuration is objectively in the object. Once it has been outputted by the designer to the object, it is there. It is "inherent" to the object, whether it is in the right context or not. A watch remains a watch, even if we find it on Mars, where nobody is using it to measure time. Why? Because we can use it to measure time, if we want. And anybody else can do the same. So, an enzyme designed to do a specific thing, designed in a context and with an awareness of the whole, still retains its functionality even if isolated from the whole. Why is this important? Because that objective functionality, even if out of context, is enough to infer design, if it is complex enough: IOWs, if a great number of specific bits of configuration are objectively necessary to make an object capable to implement that functionality. That's what I mean by "local function": just the functionality that has been given to the object by the designer. And the linked complexity allows the design inference from that object, because, while simple functionalities can be found in non designed objects (like the stone that can be used as a paper-weight), complex functionalities are never found if not in designed objects (a computer, a watch, an enzyme, and so on). I hope that we can agree on this approach, which in no way denies the truth of all the things that you have said. Thank you anyway for your deep contribution. :)gpuccio
March 20, 2017
March
03
Mar
20
20
2017
10:57 AM
10
10
57
AM
PDT
gpuccio @40: Thank you for the fulsome and helpful response. I agree that if: 1. We have a string that is somewhat repetitive, 2. The string is sufficiently long, and 3. We have a compression algorithm based on identifying and re-coding repetitive elements (rather than a dictionary-based algorithm), Then, a. We can create a compressed communication that is shorter than the original string itself, b. We can compress the string more than we could a less repetitive string, and c. The algorithm should also work in general for any string we throw at it (as long as it can recognize the characters). In that sense I agree that we can, and should, view certain strings as being more compressible than others, as a result of their more repetitive properties. ----- About my random string example, you said:
That’s OK, but obviously it is longer than simply transmitting the bits of the string itself, because those bits have however to be included in the algorithm.
Exactly. That is part of the point I have been making all along. In this kind of situation, the information involved in the setup and preparation of the compressed message -- what has to be communicated between sender and receiver -- will always be longer and include more complex specified information than the string itself. This is because in assigning an additional code on top of the string, we are adding additional information. The savings comes in when we have a sufficiently long message or many messages over time. This is the way any dictionary-based compression has to work, such as my example. Also, even a repetitive-analysis compression that could be used on any string works this way: it has to identify the repetitive element, then assign an additional coded element to that repetitive element, then transmit both the repetitive element and the additional coded element to the recipient. The initial process is always longer than just transmitting the repetitive element. The savings comes in when there is enough repetition -- either in a sufficiently large string, or in additional communications over time. ----- Now, there is another kind of "compression" possible, beyond dictionary-based or repetition-based compression. This is what you are referring to in your response to my pi example. You mentioned that for pi:
In alternative, we can simply transmit an algorithm which, according to the basic rules of the system, will generate the string as soon as received . . .
It is true that for a subset of communications we can recognize we are dealing with a mathematical calculation and, if the result of the calculation is lengthy enough, there will be a point at which communicating the calculation instruction is shorter than communicating the result. Note, however, that even in these cases we are piggybacking on a tremendous amount of background knowledge that has already been communicated to the recipient (either by us or by someone else previously), namely, what math is, how it works, what the various symbols represent, and how to calculate pi. Now, for the rare case of an infinite or near-infinite result (such as pi) the aggregate amount of information required to compute pi will end up being less than the string itself. But this case isn't so far removed from the other compression approaches I mentioned before. There is a large amount of information required up front -- more than a reasonable-length string itself. However, for a sufficiently lengthy string or for situations in which we are repeating an operation over time, we have a savings. I should add that a mathematical operation compression approach is, at some level, no different than a dictionary-based approach. In other words, we are taking advantage of the background knowledge and domain expertise of the recipient to generate the string on the receiving end. It is not that we have transmitted the string, or even that we have really transmitted the actual instructions on how to calculate the string. Rather, we have transmitted a message that someone with the correct and sufficient background information will know what to do with it. It is not completely clear to me that such situations are a legitimate form of "compression", so much as a re-coding of the string before transmission. A non-mathematical example might be helpful. We could transmit the message "write the Gettysburg Address." If the recipient has the correct and sufficient background knowledge they can re-create the string. But not because we have actually "compressed" the string. Rather, we have relied on the shared background knowledge of sender and recipient and have, instead of compressing the original string, re-coded the message to achieve the same result and then transmitted that. Telling someone to use their background knowledge to compute pi is really no different. ----- I'll have to think more about your Kolmogorov comments at the end. My hunch is you may be on to something important, but I need to think through it and understand it a bit better. ----- Beyond that, I think we've largely covered the matter of strings. However, if you'll indulge this thread for a couple more days, I'd like to return to where we started -- entropy in real physical systems, which seems to be where Swamidass' misunderstanding starts . . .Eric Anderson
March 20, 2017
March
03
Mar
20
20
2017
10:16 AM
10
10
16
AM
PDT
Origines @44: Good thoughts and examples. I don't disagree with your general thrust. I would just note that it is possible to ascertain what a part is doing, even if we don't have a top-down picture of the whole. We don't need to know everything at the highest level in order to discover and understand something at a lower level. As mentioned, I believe you are focusing on purpose, which is fine. It is also common usage and acceptable to refer to a local function in isolation. If we want to call it "activity" that is fine too. In any event, it sounds like we are all largely in agreement. Just a definitional question.Eric Anderson
March 20, 2017
March
03
Mar
20
20
2017
09:28 AM
9
09
28
AM
PDT
GPuccio #29, Thank you for your response. What follows are some general thoughts on ‘function’, ‘wholes’ and materialism. I will try to address the specific points you make later.
Eric Anderson: I think gpuccio is using both terms “function” and “system” in the standard scientific understanding of the words.
Yes and it is a good thing that GPuccio adapts, otherwise he would not be able to step in and destroy their arguments as he so frequently and convincingly does. Nevertheless, I’m of the opinion that there is something terribly wrong with this standard scientific use of those terms. Again, “random system” contains two contradictory terms and so does “local function”. IMO these terms are in place to accommodate materialistic notions and they (the materialists) are robbing us blind. Materialists are hell bound to deny the existence of wholes over and beyond the physical micro-level. But when one denies the whole one forfeits the right to use the word ‘function.’ You can’t have your cake and eat it. When one speaks of ‘function’, what one is saying is this: I see in/through/behind all these isolated parts a functionally coherent whole. On my screen I don’t just see isolated random letters, but, instead, I see a coherent whole/message. I see one thing — instead of many things. IOWs the letters you are reading are aspects of one thing. They are functionally integrated in order to convey one specific message. The letters are forced into functional subservience of a message. Notice that we can easily discern a hierarchical structure in this whole: each letter in this post is subservient to words, and the words, in turn, are subservient to sentences, which, in turn … and so forth. Further we may notice that, the higher the level the more defined things get — the more ‘differentiation’ occurs. The more integrated parts something contains the more defined something is. A single letter has arguably no parts and is (therefore) extremely flexible (undifferentiated) and can be part of a word which can mean literally anything. A word can be part of a sentence which can mean almost anything. An entire sentence can be part of a story which can mean many things. An entire paragraph is less flexible … and so forth. The idea is clear, I hope, that the higher the level, the more integrated parts, things get more and more defined, until, ultimately, the final form is stated. When we look at the Apollo 13 manual as a whole we intuitively understand that this has no bottom-up explanation. The existence of the manual cannot be causally explained from the level of the intrinsic (local?) functionality of letters. This is what Douglas Axe calls the universal design intuition. Summarizing: wholes exist well over and beyond the level of fermions and bosons, these wholes have a hierarchal structure and their internal levels have accumulative definition. And, importantly, all of this stuff cannot be explained bottom-up from the level of the parts. Returning to my point about the nature of ‘function’: in order to understand functionality one has to start at the level of the whole and work one’s way down. The function of a part can only be defined relative to the whole. Put another way: the function of a part is its role in the whole. A function is not an intrinsic property of a part, but it is an extrinsic property derived from the whole.Origenes
March 20, 2017
March
03
Mar
20
20
2017
07:51 AM
7
07
51
AM
PDT
gpuccio @41:
The fact remains that the 2000+ superfamilies in SCOP classification are the best undeniable argument for design in biology.
That seems like the most verifiable conclusion one could draw from reading that thread, despite the focus being on something else. Thank you for commenting on it. Apparently you're the most qualified person to comment on that important subject in this site. At least that's my perception, which could be inaccurate. As far as I'm concerned, I've used the clear explanations you've written on the proteins subject in order to learn more about it, hence was missing your comments in that thread. However, I understand your valid reasons for not getting into that discussion. Mile grazie!Dionisio
March 20, 2017
March
03
Mar
20
20
2017
06:50 AM
6
06
50
AM
PDT
EricMH: Thank you for the explanation. I agree with what you say, even if it is not completely clear to me how you translate the concepts of entropy ans enthalpy and free energy to the biological context. However, I think we agree on the final scenario, and that's the important thing.gpuccio
March 20, 2017
March
03
Mar
20
20
2017
06:06 AM
6
06
06
AM
PDT
Dionisio: The SUPERFAMILY database is interesting, and I am experimenting with it. Regarding that thread, the focus seemed to be on common ancestry, and frankly I am tired to discuss that. The fact remains that the 2000+ superfamilies in SCOP classification are the best undeniable argument for design in biology.gpuccio
March 20, 2017
March
03
Mar
20
20
2017
06:01 AM
6
06
01
AM
PDT
Eric Anderson: I agree with you. I agree with you on practically everything. I think we understand each other very well. :) A couple of points deserve, maybe, some further discussion. About Shannon entropy. I agree that we need a basic setup to make any discussion about that: a system were we can transmit bits, and some basic protocol to transmit them, receive them, compress them. I am not an expert of Shannon's theory, not at all, but I think that these basci "agreements" should be considered as part of the system we are analyzing. Then, inside that system, we can reason about transmission and compressibility of the intended message. What I am trying to say is that IMO you underestimate the objectivity of Shannon's entropy: as far as I can understand, it is an objective mathematical property of the message we have to transmit, even if it requires some basic agreements in the system. So, if a string is scarcely compressible, it is so because of its intrinsic properties, and not because of the way we have set the system. IOWs, a non compressible string is not compressible, however we set the system. On the contrary, a highly ordered string is always compressible, whatever the system You offer a counter-argument when you say: "There is no reason I can’t write an algorithm that contains a function that says “if I come across aljgoqjektnanksautyapley, then transmit the symbol &.” Ta da! Under the new “Anderson Method” of compressibility, the string aljgoqjektnanksautyapley turns out to be extremely and highly compressible — it can be transmitted with a single character!" I think that here you are making an error that deserves to be discussed. So, let's say that we have our basic settings for transmission and compression. Those settings are part of the system, so they can work for any string I want to transmit. Under those basic conditions, let's say that the string you propose: aljgoqjektnanksautyapley is random and completely not compressible. Therefore, if you want to transmit it, the only way is to give all the bits that describe it. Now, if you want, instead, to transmit an algorithm that can generate the string, that is an acceptable way to transmit the string itself. The algorithm you propose is to transmit: “if I come across aljgoqjektnanksautyapley, then transmit the symbol &.” And then transmit the single character &. That's OK, but obviously it is longer than simply transmitting the bits of the string itself, because those bits have however to be included in the algorithm. IOWs, the complexity of the "compressing algorithm" is higher than the complexity of the object we want to compress. So, the Kolmogorov complexity is the complexity of the string itself, because it is simpler than the complexity of the algoritthm. Now, let's make another example: the digits of pi. Let's say that the message we want to transmit is the first 100 digits of pi. Of course, here we are not interested in communicating what they are or what they mean as a mathematical constant. We just want to transmit the sequence. If we assume that such a sequence, even if certainly not random, is scarcely compressible (which I believe should be true), Its Shannon entropy will be very similar to the number of bits necessary to describe it. In alternative, we can simply transmit an algorithm which, according to the basic rules of the system, will generate the string as soon as received (just think in terms of a general Turing machine). Well, that algorithm will probably be longer than the string itself. So, we are apparently in the same situation as before: no real compression. But what is we want to transmit the first 10^15 digits of pi? In that case, an algorithm that can compute them is probably much shorter than the string itself. So, in that case, the Kolmogorov complexity of what we want to transmit would definitely be the complexity of the algorithm, and not the complexity of the string itself. As you can see, these are objective and very interesting properties of strings as such, and they do not depend on the basic settings of the system, even if they do require a basic setting of the system. I will try to summarize the above concepts: a) Some strings are scarcely compressible because of their intrinsic nature. In this group we find most purely random strings, but also many functional strings where the function does not require some special order in the string, but only the correspondence of the string to a higher level of understanding (for example, how a proteins folds and works at biochemical level, or how an ordering algorithm works). In general, I think that functional strings are more ordered than a purely random string (that's why we can sometimes recognize a language even if we don't know what it means). But in some cases that "intrinsic order" can be very low, and I think that protein strings are usually of that kind. b) Some strings are highly compressible because they are highly ordered. A string of 10^15 "1". or of 10^15 "01" is a good example. c) Other strings are compressible because, even if they are not ordered at all, they are computable. The sequence of digits of pi is a good example. d) For computable strings, here is an interesting aspect: the Kolmogorov complexity (which is what we are really interested in computing functional information) grows as the string becomes longer, but as soon as the complexity of the string equals the complexity of the algorithm that can compute it, the Kolmogorov complexity remains the same, while the length of the string continues to increase. That is very interesting, because the generating algorithm can be viewed as the specification of the string (after all, it can compute the string because it has been built using the information about what the string really is, in the case of pi the ratio of a circumference to its diameter). On the contrary, the length of the string is related to its potential complexity. So, if we consider simply the potential complexity, we have the strange result that the complexity can increase at will in a system that includes the generating algorithm. If we equal that potential complexity to the functional complexity, we would have the paradox of generating new functional complexity without any further conscious intervention in the system. But luckily that is not the case. Functional complexity corresponds to the functional Kolmogorov complexity of an object, not to the functional potential complexity. In the case of pi, the complexity apparently increases as the computed result becomes longer, but the specification (the generating algorithm) remains the same. So, the functional complexity in the system (the functional Kolmogorov complexity) remains the same, even if a constantly growing string corresponding to the specification (the ratio of a circumference to its diameter) is being generated in the system. OK, I apologize for this long and probably too complex discussion. I hope you will be patient enough to consider it.gpuccio
March 20, 2017
March
03
Mar
20
20
2017
02:31 AM
2
02
31
AM
PDT
Eric Anderson and Origenes: Wow! This has really become an interesting discussion! :) Many thanks to you both for your precious contributions. Eric, I will comment on your thoughts in next post. For the moment, I would like to say that you have stolen my thoughts with this statement: "That said, I understand where you are coming from Origenes. What you are describing as function sounds like something we might call “purpose”." Well, you may believe it or not, but that is exactly what I was going to write to Origenes in my last post to him. In the end, I refrained, probably for fear of being misunderstood, or of opening a much wider discussion. But, as you said it yourself, I can happily agree. I absolutely agree also with all the further comments you offer in post #32. That's exactly what I think. I will only add that purpose is probably more than function, but that any wider purpose has to be implemented by specific functions. All discussions about purpose are extremely important and necessary. The only problem I see is that, like many basic philosophical problems, it could lead to some form of infinite (or at least very long) regress, where we end up asking the purpose of life, or of the universe itself. Nothing bad in that, after all. :)gpuccio
March 20, 2017
March
03
Mar
20
20
2017
01:49 AM
1
01
49
AM
PDT
Eric Anderson @32:
I believe gpuccio has been focusing his research on identifying complex specified information at a more basic level than ultimate purpose. [...] it has the advantage of being more mathematically tractable and of setting a lower limit to what must be required for any higher level of functional organization.
Yes, agree. Perhaps this relates to the comment @35?Dionisio
March 19, 2017
March
03
Mar
19
19
2017
11:54 PM
11
11
54
PM
PDT
gpuccio: Off topic: I expected to read your insightful comments in this thread: https://uncommondescent.com/informatics/common-ancestry-bioinformaticist-julian-gough-on-the-superfamily-database-on-proteins-in-genomes/ Did I have wrong expectations? :)Dionisio
March 19, 2017
March
03
Mar
19
19
2017
11:42 PM
11
11
42
PM
PDT
The term 'system' used in biology: https://en.wikipedia.org/wiki/Biological_system The same term in other widely known contexts: https://en.wikipedia.org/wiki/Operating_system https://en.wikipedia.org/wiki/Economic_system https://en.wikipedia.org/wiki/Political_system http://www.thefreedictionary.com/legal+systemDionisio
March 19, 2017
March
03
Mar
19
19
2017
11:09 PM
11
11
09
PM
PDT
gpuccio @24:
[...] we can focus our attention, for practical reasons, on the “local” function of the protein, without considering, for the moment, the links to other levels of function in the cell.
That seems a very practical approach that is conducive to 'bit' quantification of structural novelties at the individual protein level. Higher functionality levels may be more difficult to quantify.Dionisio
March 19, 2017
March
03
Mar
19
19
2017
10:56 PM
10
10
56
PM
PDT
@GPuccio, in biology, as entropy increases the chance of DNA being functional decreases, and CSI decreases. High entropy only means high CSI if it manages to hit a small, specified target. Alternatively, if increase in entropy invariably meant an increase in functionality, then this means the functional search landscape is extremely dense. An upshot of this is that genetic engineering would be quite easy. We can create random DNA sequences much more rapidly than nature, and should be able to easily hit the functional targets in the lab. However, this would still not equate to high CSI, since specificity is extremely large. The only way that arbitrary entropy increase can equate to functionality increase in a sparse functional landscape is if there are pathways to the functionality. But this requires large amounts of prior CSI in the construction of these pathways, and the entropy increase is not a contributing factor to the CSI. So, in whatever way it is correct to say entropy increase also increases functionality, entropy increase cannot equate to CSI increase. The two concepts are antithetical to each other by definition. CSI itself is not an alien concept invented by the ID movement. The concept shows up in many different disciplines, such as free energy in physics as I pointed out. But, it always introduces the same problem of being inexplicable in terms of chance and necessity, and leads to nebulous work arounds such as emergence and self organization. This is merely a matter of math, because by definition chance and necessity cannot produce CSI. All they can do is destroy CSI, and the 2nd law of thermodynamics with free energy is one example of this.EricMH
March 19, 2017
March
03
Mar
19
19
2017
10:37 PM
10
10
37
PM
PDT
gpuccio: One other quick point: Let's take a random string: aljgoqjektnanksautyapley There is no reason I can't write an algorithm that contains a function that says "if I come across aljgoqjektnanksautyapley, then transmit the symbol &." Ta da! Under the new "Anderson Method" of compressibility, the string aljgoqjektnanksautyapley turns out to be extremely and highly compressible -- it can be transmitted with a single character! :) And if that were the extent of my algorithm, then when it came across the string "to be or not to be" it would have no way to compress it and would have to transmit the entire string. There is nothing magical about compression or compression algorithms. An algorithm is just built upon a statistical expectation of what it expects to see, given various parameters about anticipated types of messages, anticipated language, grammar, etc. In practice what makes a meaningful phrase more compressible than a random string is the understanding and expectations and skill of the algorithm writer in creating a good algorithm for the task at hand. We cannot simply "take for granted" that all the work and effort and understanding and information required for compression somehow exists in and of itself and that a given string contains more or less information based on its compressibility due to the intelligence and hard work of the programmer. ----- Based on our understanding of how language and sensical information arises and is encoded, we can draw a general observation that, in a given symbolic system, complex specified information generally lies between the extremes of order and randomness. But beyond very specific, well-defined examples we can't draw any general conclusions about having more or less information as we move away from a sense string and toward one or the other end of the spectrum. If my sense string were modified to be a string of random characters it would have no more information than if it were to decay to a string of repeating 'a's. The information content cannot be ascertained by compressibility. (I know you know this. I'm just brainstorming out loud here . . .)Eric Anderson
March 19, 2017
March
03
Mar
19
19
2017
06:23 PM
6
06
23
PM
PDT
gpuccio, Origenes -- good discussion. If I may jump in, I think gpuccio is using both terms "function" and "system" in the standard scientific understanding of the words. That said, I understand where you are coming from Origenes. What you are describing as function sounds like something we might call "purpose". I'm reminded of Aristotle's four causes: the material cause, the formal cause, the moving cause, and the final cause (or purpose). We can certainly analyze biology across more than one of these. I believe gpuccio has been focusing his research on identifying complex specified information at a more basic level than ultimate purpose. This is a bit less accessible, perhaps, than a more macro view, but it has the advantage of being more mathematically tractable and of setting a lower limit to what must be required for any higher level of functional organization.Eric Anderson
March 19, 2017
March
03
Mar
19
19
2017
05:00 PM
5
05
00
PM
PDT
gpuccio @19:
You example about pi is not appropriate, IMO. “write the first 100 digits of pi” is not a way of conveying the first 100 digits of pi. IOWs, it is not a “compression” of the first 100 digits of pi.
I apologize if my example wasn’t clear. I agree with you that for purposes of transmission we are interested in a compression algorithm. Let me press, however, on the example a bit, because you are selling it short too early and the example remains instructive, even insightful, if we think through it all the way.
But just giving the instruction: “write the first 100 digits of pi” is not good, because the instruction does not have the true information to do what it requires.
I agree that the transmitted instruction does not have within itself the information to do what is required. But that is exactly my point. When we are dealing with a compression algorithm, the transmitted message never has the information to do what is required. It is always the case that we have some kind of protocol or code or pre-agreed-upon algorithm to unpack the transmission and recreate the original string. I can most certainly construct a compression algorithm that would recognize the first 100 digits of pi and then if it comes across that, transmit the instruction “write the first 100 digits of pi”. This is no different than transmitting a mathematical instruction to compute pi, which itself has to be understood and run at the other end.
On the other hand, the instruction “write 01 50 times” is a true compression (provided that the simple instructions are understood as part of the basic transmission code).
Exactly. Having the instructions understood on the receiving end is always the issue. It is no different than my example of pi. I can compress and uncompress a string any way I want, as long as the instruction on both ends is understood. But there is no free lunch and whenever I transmit a compressed signal I have to also transmit the protocol or code or unpacking instruction (either contemporaneously or beforehand). The upshot of this fact which most people don’t recognize is this: For any reasonable-length message it typically takes more bandwidth to implement a code, compress the string, and transmit both of them than it would require to just transmit the uncompressed string in the first place. There is no inherent reduction in the quantity of what needs to be transmitted, just because I can think of ways to compress the string. Instead, the savings comes in when we have a very long message (e.g., a book versus a sentence) or when we do lots of transmissions over time. -----
Shannon’s theory is exactly about “reduction of uncertainty” and according to Shannon’s theory the values of entropy are different for different messages. That does not depend on external factors, but on intrinsic properties of the message sequence.
I think I understand where you are headed, but let me make sure. In order to implement any algorithm or to calculate the Shannon measure we must, as you mentioned at the beginning, have the code in place beforehand. In addition, we have to understand various parameters, such as the language we are dealing with, the likelihood of occurrence of particular symbols, etc. All of these things are external factors that impact how the algorithm operates and what kind of result we get, are they not?
However, that has something to do with order (ordered systems being more compressible, and easier to describe) . . .
I don’t dispute that a more ordered system – in the sense of more repetitive – would be more compressible. But, again, we have to ask ourselves how and why? It is more compressible because (1) we as intelligent beings are capable of recognizing the repetition, (2) we have layered a new code on top of the existing string, replacing repetitive elements with shorter elements and instructions to recreate the repetitive elements, (3) we have transmitted the code and the unpacking protocols to the recipient. Once all of those are in place then, yes, we can compress a repetitive string more than a less repetitive string. This is all well and good for what Shannon was interested in: namely faithful transmission and compression of the transmission. But when people like Dr. Swamidass start talking about information and hinting that we have more information because a string is less repetitive or less compressible, then something is amiss with their understanding.
. . . in discussions about Shannon entropy, that “code” is given for granted, and is not part of the discussion.
That it is “taken for granted” is precisely part of the problem when we talk about strings and, in particular, when we talk about Shannon. In the first place, when talking about Shannon we aren’t really dealing with information. But there is a second, more nuanced, issue I am trying to highlight in my last couple of comments to this thread. Namely, that there is a great deal of information being smuggled in behind the scenes. It is this additional information, operating behind the scenes, that gives us the impression that we have somehow compressed things and ended up with less information. No. That is not how compression works. What we have in fact done is add an additional layer of information – highly complex, specified information – on top of our string. And a more random string does not mean that it has inherently more information somehow. It is just less compressible or requires a better algorithm. -----
As I have tried to argue, function is often independent from intrinsic order of the system. The sequence of AAs for a functional protein may be almost indistinguishable from a random AA sequence, until we build the protein and see what it can do.
Agreed. Order, in terms of repetitiveness, is not equivalent to function. I fully agree there. To go the next step, I would add that complex specified information resides between the extremes of order and randomness, existing sometimes on a knife’s edge balancing act between contingency and order.
Now, I think that we should use “complexity”, in a generic and basic meaning, just as a measure of “the quantity of bits that are necessary to do something”.
I definitely agree that we can use the word in a generic and basic meaning. In fact, I would prefer a more basic definition than layering on a bit calculation. Yes, there are different ways to measure complexity and doing a bit calculation per Shannon or a modified Shannon is probably a reasonable surrogate to help us measure the complexity of a system. But I think we can talk about complexity in a generic sense, before determining which system we are going to use to measure it.
“Shannon entropy provides an absolute limit on the best possible average length of lossless encoding or compression of an information source.” So, this kind of “complexity” has nothing to do with meaning or function.
Absolutely agree. As did Shannon. :)
On the contrary, the “complexity” related to functional information measures the bits that are necessary to implement a function. As I have said, this is a concept completely different from compressibility and transmission. As functional information is often scarcely compressible, the functional complexity of a system will often be similar to the total complexity of a random system.
I’d like to understand a bit better what you mean by this. If we are going to implement a function in three-dimensional space we can go about it a couple of different ways. 1. We can specify the location and trajectory of each particle in the system. 2. We can use start points, end points, vectors, axes, geometry and math to describe the system. A CAD file would be an example of utilizing this kind of approach. (I think you’ve use this example before.) The second approach would typically be more compressible than the first, would it not? Whether it is rational to do #1 for a non-functional system and #2 for a functional system is a separate question I’ve been addressing in my comments. But in general, #2 would often be significantly more compressible than #1, correct?
In this case, we are rather interested in finding how much of the potential complexity of the system is really necessary to implement its function. That’s what we do by computing the rate of the target space to the search space. But that has nothing to do with compression based on order.
I’ll have to think about this some more. You’ve probably already done so, but can you give an example of what you mean here?
I think that too much importance is often given to terms like “complexity” and “information”. The only meaning of “information” that is interesting for ID is specified information, and in particular functionally specified information. All other meanings of “information” are irrelevant.
I think it would be difficult to give too much importance to information, as that is at the basis of everything we are talking about, but I understand what you are saying about definitional battles, and we are on the same page substantively. As a practical matter, I prefer not to give away the word to the detractors. I don’t want people thinking there are bunch of different kinds of information out there, and intelligent design just deals with one particularly esoteric kind. Then we are forced to have a subsequent battle as to whether our “information” is more important or more relevant to biology than their “information.” It is too easy for people who don’t understand design or who are actively critical of design, like Swamidass or Liddle, to jump on something like Shannon “information” without understanding what they are talking about and claim that it is the real information and that those ID folks are trying to push some strange idea called complex specified information. No. When I talk about intelligent design I am talking about information in the basic, understandable, dictionary meaning of the word. I am dealing with real information. And anyone talking about Shannon’s measure or claiming that information increases with disorder isn’t talking about information. In fact, they don’t know what they are talking about. So I am not willing to give away the word “information” to anyone who comes up with a nonsensical and illogical use of the word. I will continue to try to educate people on what information is, one step at a time. Hopefully it will help someone to actually understand the issues. All that said, I realize this is something of a tactical decision. I certainly understand and respect your approach, which I have taken myself at times. At the end of the day you are certainly correct that we are interested in complex specified information. Hopefully people will be willing to make the effort to understand what that is.Eric Anderson
March 19, 2017
March
03
Mar
19
19
2017
04:44 PM
4
04
44
PM
PDT
... or maybe not. :) :)Upright BiPed
March 19, 2017
March
03
Mar
19
19
2017
04:11 PM
4
04
11
PM
PDT
Origenes: OK, it's only a question of words, but words have some relevance to the reasonings, here, as I will try to explain. My concept of function is much simpler. If you look at my OP about functional complexity: https://uncommondescent.com/intelligent-design/functional-information-defined/ you will see that I define function in a very simple way: anything that you can do with the object you are observing. The only requirement for a function is an explicit definition of the function itself. So, I can define for a stone the function of paper-weight, or just the function of being a rudimentary weapon, and so on. There is no concept of "serving" something else, just the concept of what can be done with the object in its current configuration. This empiric definition of function is vital, IMO, for the whole ID theory, because it is the only one that allows an objective use of the concept, and allows specific measurements of the complexity linked to each defined function. If you have read some of my posts here, you can understand that such an explicit and objective definition is vital for the reasonings I make here. In a sense, any scientific discourse must be done in what you call "a fitting context". No concept of science has any meaning if there are not people who can understand it, for example. But objective science must consider observable facts that are in some way independent from who makes the observation. So, an enzyme does what it does in the same way either in the cell or in the lab. That is a fact. So, I call function what it does, because it is something that can be done with the object, and it is objectively so. There is more: the reason why an enzyme does what it does lies in its specific configuration, which in the end depends on the AA sequence. Now, even if you consider that an enzyme does what is does because it "serves" the cell in some way, still it is able to do it exactly for the same reason that it is able to do the same thing in the lab: its configuration. When the enzyme catalyzes a reaction, it has usually no active part in what happens after, with the results of its activity. Therefore, the information in its configuration is essentially necessary to catalyze the reaction, and, usually, for nothing else. Therefore, it is obvious that we must use the "local function", as I call it, to measure the complexity linked to the configuration. The specific bits of functional information present in the sequence are necessary to catalyze the reaction, and not for anything else. The "serving" to the cell is due to other configurations, like the presence in the right context of many molecules that, for example, work in a cascade. But each molecule does what it does. The total result is the sum of the different "activities" (as you call them), but the intelligent information in each molecule is finalized to the specific activity, while the intelligent information in the design of the whole system (including all the molecules) is what ensures the final result. So, my use of function to define what an object does, by itself, is not only a verbal choice: it is a necessity. It is that "local function" that is supported by the configuration, and therefore the AA sequence. That is the digital information that we want to measure, because it measures the improbability of having that single object without a designed process. Of course, if we want to consider the functional information of the whole system (for example a protein cascade) we have to sum the functional complexity (in bits) of each component, and also to consider the information necessary to have all the components at the right place, at the right time, and so on. Dembski has tried something similar, but the task is really difficult, and anyway it has to rely in part on the measurement of the functional information in each component. So, I am in no way underestimating the importance of considering whole systems, and the functional information necessary to have them as a whole. I am only saying that all starts at the level of each single component, and each single component has its local function and is configured for that local function. That function is as much a function as the global function of the whole system, or of the cell, or of the organism. Because, in the end, the only simple and objective way to define a function for an object is: what can we do with this object? However, I absolutely respect your terminology and your way of thinking. I just wanted to explain the operative reasons that make me stick to mine.gpuccio
March 19, 2017
March
03
Mar
19
19
2017
04:05 PM
4
04
05
PM
PDT
GPuccio: Thank you for your response.
For local function I mean a well defined function that the object can implement by itself, even if it is not connected to a wider context.
Instead of the term ‘local function’ I would suggest ‘activity’ or ‘operation’. I prefer ‘function’ to refer to the state of being subservient to something larger. A heart is functional / subservient to the organism. A function in this sense can never be ‘local’; can never be isolated from a larger context.
I have given the example of an enzymatic function. Even is an enzyme can be part of a wider context, and indeed it always is, you can take the protein, study it in the lab, and still it can implement its enzymatic activity, even if it is isolated by its natural context.
First, you still need to provide a context for the enzyme to implement its enzymatic activity. Second, in this new unnatural context it obviously cannot have the same function — because the cell is absent and function is context dependent — but it can have the same activity. Maybe we can find agreement here. I think that saying that the enzyme has the same activity in the cell and in the lab resolves my concern. For me it is incoherent to say that the enzyme has the same function — or ‘local function’ — in the cell and in the lab. You are free, of course, to continue to use the term ‘local function’. However I find that term incoherent — those two words contradict each other — and I will translate ‘local function’ with ‘activity’.
Indeed, I don’t understand why you say that “X is not functional if you cannot point out something larger than X.”
X is functional for something else. X has to serve, be useful, for something larger in order to be rightly termed “functional”. That’s what functional means.
How can that be true? Isn’t an isolated enzyme functional, for its ability to accelerate a specific reaction?
No, I believe it isn’t. Its activity is only functional if it serves a larger coherent context.
Isn’t a single phrase functional, if it conveys a specific meaning?
A phrase can only convey a meaning if it is situated in a fitting context of e.g. the communication between two persons. IOWs you are assuming a fitting context here. A single phrase on a piece of paper in the unfitting context of e.g. the uninhibited planet Draugr obviously conveys no meaning and therefore has no function.Origenes
March 19, 2017
March
03
Mar
19
19
2017
02:55 PM
2
02
55
PM
PDT
Hello Origenes, GP, A paper you might find interesting for background. The Nature of Hierarchical Controls in Living Matter The paper appears below the fold.Upright BiPed
March 19, 2017
March
03
Mar
19
19
2017
12:54 PM
12
12
54
PM
PDT
Origenes: For local function I mean a well defined function that the object can implement by itself, even if it is not connected to a wider context. I have given the example of an enzymatic function. Even is an enzyme can be part of a wider context, and indeed it always is, you can take the protein, study it in the lab, and still it can implement its enzymatic activity, even if it is isolated by its natural context. The same is true for a phrase that has a definite meaning. If I say: "The molecule of water is made of atoms of hydrogen and oxygen". I am conveying a specific meaning. Of course, that can be only the beginning of a wider discourse about water, or about many other things, but still the "local" meaning of the phrase remains. So, I would define "local function" as any function that the object we are analyzing can implement by itself, without any reference to a wider context. Indeed, I don't understand why you say that "X is not functional if you cannot point out something larger than X." How can that be true? Isn't an isolated enzyme functional, for its ability to accelerate a specific reaction? Isn't a single phrase functional, if it conveys a specific meaning? Another example could be a simple ordering algorithm, which orders the inputs it receives. It can certainly be part of a much more complex software, and indeed it almost certainly is, but can you deny that the code of the ordering algorithm, even if isolated, can still order inputs? I would like to understand better why you apparently don't accept this idea.gpuccio
March 19, 2017
March
03
Mar
19
19
2017
12:01 PM
12
12
01
PM
PDT
Gpuccio, Thank you for your response. IMHO the term ‘function’ is always connected to a higher level — a larger whole or a ‘context’. X is not functional if you cannot point out something larger than X. IOWs X can be functional only if X is a component of something larger than X. Function implies hierarchy. Function only exists if there is a larger whole to serve.
GPuccio: However, that said, it is also true that we can focus our attention, for practical reasons, on the “local” function of the protein, without considering, for the moment, the links to other levels of function in the cell.
Could one argue that “local function” refers to the adjacent higher level of function? As in, the local function of a letter is being part of a word (which represents a higher level function), and the local function of a word is being part of a sentence (which represents a yet higher level function) and so forth?
GPuccio: Finally, I generally use the word “system” in a generic sense, not connected to functionality, because that is the common use of the word in physics. See Wikipedia: ...
Yes I understand. Nevertheless I would have preferred them to use another term. For me 'system' is connected to 'design' and 'function', so this choice of terms is mildly confusing.Origenes
March 19, 2017
March
03
Mar
19
19
2017
10:51 AM
10
10
51
AM
PDT
1 2

Leave a Reply