He writes, “Earlier this year, I published a three-part exchange with Querius (linked below), who is looking for answers as to whether computers can someday think like people. He contacted me again recently and once again kindly gave me permission to publish our discussion”:
… because the halting problem, is undecidable, even with unbounded resources, then the AI still cannot perform like a human mind. That seems counterintuitive at first because all the halting programs will eventually halt, given enough time and resources. However, because the AI does not know ahead of time how many halting programs there are, it will never know when they’ve all stopped. Instead, we’ll end up with a permanently frozen world when the AI goes head to head with a human.
Eric Holloway, “COULD AI THINK LIKE A HUMAN, GIVEN INFINITE RESOURCES?” at Mind Matters News