Artificial Intelligence Intelligent Design Mind Neuroscience

Eric Holloway: Can computer neural networks learn better than human neurons?

Spread the love

Holloway says, they can and do; when artificial intelligence programmers stopped trying to copy the human neuron, they made much better progress. Humans can do things that AI cannot do, as we saw earlier, but those abilities are not due to the superior learning ability of a human neuron:

Brain neurons operate on what is called the “all or nothing” principle. When enough charge is built up by the neuron’s synapses, it neuron fires. But until then, the neuron does absolutely nothing. The neuron can thus be seen as an on–off switch. It is either firing, or it is not. There is no in-between stage. This makes learning difficult.

To understand why the “all or nothing” principle makes learning difficult, think about playing the “hot or cold” search game where you are searching a room for a treasure. A helpful bystander says “Hotter!” as you get closer to the treasure and “Colder!” as you get farther from it. This signal is very effective in helping you locate the treasure.

But what if we tweak the game. Now, the bystander tells you only whether you have found the treasure or not. If you find the treasure, the bystander says “Yes.” If you did not find the treasure, the bystander says “No.” This new version will take quite a bit longer to play because the bystander is really not providing any information that speeds up the game.

Eric Holloway, “Can computer neural networks learn better than human neurons?” at Mind Matters News

Takehome: Humans can do things that AI cannot do, as we saw earlier, but those abilities are not due to the superior learning ability of a human neuron.

You may also wish to read: Artificial neural networks can show that the mind isn’t the brain Because artificial neural networks are a better version of the brain, whatever neural networks cannot do, the brain cannot do. The human mind can do tasks that an artificial neural network (ANN) cannot. Because the brain works like an ANN, the mind cannot just be what the brain does. (Eric Holloway)

29 Replies to “Eric Holloway: Can computer neural networks learn better than human neurons?

  1. 1
    Bob O'H says:

    There’s a remarkable lack in that piece of any discussion of how humans learn. It’s also undermined by the fact that artificial neural nets are run on computers that fundamentally operate only with “all or nothing” instructions (otherwise known as “on and off”, or “TRUE and FALSE).

  2. 2
    Alan Fox says:

    Brain neurons operate on what is called the “all or nothing” principle.

    Seems simplistic. Two (at least) variables are being ignored: the rate at which a neuron fires (the literature is inconclusive on whether individual neurons vary on rate or whether this is an effect of averaging) and the threshold at which a stimulus provokes a neuron to fire (literature indicates first stimulus will trigger firing at a lower threshold than subsequent stimuli).

  3. 3
    ET says:

    Alan quote-mines and then attacks a strawman. It must be Monday.

  4. 4
    Alan Fox says:

    Joe, do you never tire of your content-free knee jerks? What a wonderful mascot you are for the UD community.

  5. 5
    ET says:

    Alan the hypocrite strikes again! All Alan posts are content-free knee jerks. You are nothing but a punk, Alan. You are a sad example of humanity.

  6. 6
    Bob O'H says:

    Alan – to be fair, artificial neural nets also include a threshold, so that aspect is equivalent.

  7. 7
    relatd says:

    BOH at 6,

    The average person does not discuss neural networks – ever. “Hey Bob. What do you think of neural nets?”

    What?

  8. 8
    relatd says:

    ET at 5,

    Relax. Alan has an assignment to complete. “Go to UD and disrupt it. Make noise. Distract people.”

    Typical example of Protest Culture.

  9. 9
    Querius says:

    Relatd @8,

    Exactly! And the next assignment is to go and shout “squirrel” at a dog show . . .

    When considering any machine, including a neural network, the following is true:

    1. Don’t worship what you just made.
    2. It’s a tool that leverages human creativity and genius.
    3. A shovel works better than digging with your hands like a dog.
    4. The shovel is your tool, not the other way around.

    -Q

  10. 10
    asauber says:

    “4. The shovel is your tool, not the other way around.”

    I used to volunteer teach a computer class at a community center for seniors years ago and I realized right away that one of the recurring themes had to be the “the computer is your tool, and you are not the tool of the computer.” The Philosophy of Computer Use should have been in the title of my presentation.

    Andrew

  11. 11
    Bob O'H says:

    Relatd @ 7 – in my line of work I might actually be asked that. I guess this means I’m not average.

  12. 12
    asauber says:

    “I guess this means I’m not average.”

    Bob O’H,

    Tru. I’ve always suspected you might be a little below average. 😉

    Andrew

  13. 13
    relatd says:

    Andrew at 10,

    What? A computer is nothing more than a buying and selling, with some information content, tool. It was designed by our secretive friends at the Defense Advanced Research Projects Agency, or DARPA. ARPA-Net was designed to allow scientists to communicate secret data over a secure network. The military went on to develop this network further but released a modified version into the wild. We are typing into it now.

  14. 14
    relatd says:

    Querius at 9,

    Shouting squirrel at a dog show. I can picture that.

  15. 15
    Bob O'H says:

    asauber @ 12 – hey, that’s mean!

  16. 16
    relatd says:

    BOH at 11,

    Most people don’t understand science and some just aren’t interested. As a specialist myself, I get questions from average people that fall into the vaguely curious category. Otherwise, if neural nets don’t have an immediate impact on their lives, they generally don’t retain any information about it.

  17. 17
    Alan Fox says:

    Alan – to be fair, artificial neural nets also include a threshold, so that aspect is equivalent.

    Bob, your fairness in the face of *untyped expression that I’m now attempting to transmit telepathically* never ceases to impress me. You set a standard to which I’d like to aspire.

    So are you saying that AI programs now exist where input signals emulate stimuli, where threshold to on from off can vary? Is the variance linked to another parameter?

  18. 18
    asauber says:

    “A computer is nothing more than a buying and selling, with some information content, tool.”

    Relatd,

    Yes, but not everyone is taught that. And who teaches it to seniors?

    Andrew

  19. 19
    relatd says:

    Andrew at 18,

    As a person who is likely just a little older than you, I’d like to point out that some seniors could care less that computers exist much less how to use them. I was in contact with a lady who was 70. She got along fine without a computer.

  20. 20
    asauber says:

    “As a person who is likely just a little older than you, I’d like to point out that some seniors could care less that computers exist much less how to use them.”

    Relatd,

    What about the ones who sign up for computer classes?

    Andrew

  21. 21
    Querius says:

    Bob O’H @17,

    asauber @ 12 – hey, that’s mean!

    Yes, the mean is the average. But is it the arithmetic or geometric mean? ;D

    Bob O’H@6,
    And after you’re finished explaining machine learning and filters in detail to Alan Fox, could also answer the question of whether the Pope is Catholic and provide a summary of the history of the world?

    -Q

  22. 22
    Seversky says:

    I enjoy playing with computers but, to quote Mr Spock from the ST:TOS episode The Ultimate Computer”, “Computers make excellent and efficient servants, but I have no wish to serve under them. “

  23. 23
    EDTA says:

    I don’t want us to get too imprecise here. We seem to be losing some details we should take into account:

    >Brain neurons operate on what is called the “all or nothing” principle. When enough charge is built up by the neuron’s synapses, it neuron fires.

    But at a slightly higher level, its firing _rate_ is what matters. That’s more sophistication than a mere on/off switch.

    >Because artificial neural networks are a better version of the brain,…

    I do want to see actual measurements here. Better in what quantitative measure? Speed of training? Amount of information captured from the problem domain? Accuracy of learning?

    From the mindmatters.ai article:

    >Another difference is that the brain relies on lots of feedback between neurons but neural networks tend to get little feedback.

    Except for recurrent neural networks, which depend on feedback.

    >Neurons have at best three digits of precision, while neural networks have unlimited precision.

    Actual artificial neural networks are limited by the precision of the underlying floating point number representations. That’s not unlimited precision.

    >Typically, each neuron is connected to just a few other neurons but in neural networks each node is connected to many other nodes.

    A single human neuron may connect to as many as 10K other synapses in the brain.

    I’d like to see a lot more care taken in what claims we are making here.

  24. 24
    Querius says:

    EDTA @23,

    I do want to see actual measurements here. Better in what quantitative measure? Speed of training? Amount of information captured from the problem domain? Accuracy of learning?

    Great points!

    Actual artificial neural networks are limited by the precision of the underlying floating point number representations. That’s not unlimited precision.

    Yep. But precision is not the same as accuracy, nor is it the same as a concept. “Mr. Spock” might have been able to tell you that a collision will occur in 4.78336590812 seconds, but that’s much less useful than “Accelerate like hell NOW!”

    A single human neuron may connect to as many as 10K other synapses in the brain. I’d like to see a lot more care taken in what claims we are making here.

    Indeed! Here’s a quote from a recent press release on artificial synapses with an illuminating admission:

    Despite the runaway success {i.e. mildly helpful in some cases, very impressive success in the games chess and Go} of deep learning over the past decade, this brain-inspired approach to AI faces the challenge that it is running on hardware that bears little resemblance to real brains. This is a big part of the reason why a human brain weighing just three pounds can pick up new tasks in seconds using the same amount of power as a light bulb, while training the largest neural networks takes weeks, megawatt hours of electricity, and racks of specialized processors.

    -Q

  25. 25
    EricMH says:

    What I’ve noticed in my research is artificial neural networks become less like the brain the more effective they are. That indicates to me the brain structure is not effective for algorithmic learning.

    Now it is true that computer circuits are binary, like the perceptron. And computer circuits are highly effective. But, computer circuits are programmed by people, not by back propagation. They’ve tried to train differentiable Turing machines, but the training doesn’t go anywhere.

    Which leads to an interesting point. Both brains and computer circuits are binary. The latter are programmed to great effect by humans. Maybe what’s happening in the brain is similar, the human mind is programming the brain’s logical structure. In fact, the original Mcculloch and Pitts neuron model that inspired neural networks was intended to show the brain embedded logical concepts in the neuron structure.

    My next piece will expand on this point, showing why neural networks fail at medical decision making for precisely this reason: they can’t discover logical principles while humans can.

  26. 26
    Querius says:

    EricMH @25,

    My next piece will expand on this point, showing why neural networks fail at medical decision making for precisely this reason: they can’t discover logical principles while humans can.

    Thank you, that will be interesting! What occurs to me is that the nature of natural information is fundamentally different than how computers deal with digital data. This difference may be the source of the failures you’re referencing.

    In my experience, information has a tree-like structure of successive abstraction in one “direction” and successive elaboration/detail in the other. A solution to a problem might exist in either direction: either in general principles or in arcane details/discoveries/relationships.

    -Q

  27. 27
    Alan Fox says:

    Hi Eric
    Is your research limited to computer modelling or do you do any neuroscience?

  28. 28
    EricMH says:

    @Alan Fox, I don’t do any neuroscience. For my research I’ve been reading articles on Wikipedia and Google scholar to trace the development of neural networks.

  29. 29
    EricMH says:

    @Alan Fox, regarding the spike trains generated by neurons, engineers have tried that idea in neural networks. But, as per my point about not be differential, that leads to not very trainable neural networks.

    https://en.wikipedia.org/wiki/Spiking_neural_network#Applications

Leave a Reply