upvote
A software cannot be intelligent.

This is very easily proven by a mind experiment where you replace all the transistors by billions of humans calculating the same software output.

They will not create a new intelligence by doing so.

reply
Run the same thought experiment the other direction: at a low level, the human brain is just physics and chemical mechanisms, which in principle could be perfectly imitated by transistors calculating the same output. (Yes, such a thing isn't realistic, but neither is billions of humans doing perfect matrix math by hand.)

I still haven't seen a robust definition of "intelligence" that would allow me to tell the difference. (Bear in mind, this is a definitional struggle even in biology: if we were walk back the evolutionary chain from a human, to a protozoan, it's not clear that you could pick a single point where a non-intelligent creature gave birth to an intelligent one. It's a Paradox of the Heap.)

reply
No. There is serious scientific evidence already that our brain functions on the level of quantum mechanics. Human brain is not just an automata; wavefunction collapse may be very essential to our concept of free will.
reply
I haven't dug into these claims in detail; I take seriously a possible relationship between quantum mechanics and the brain. But "free will" is hardly a settled question either (I'm closer to the "compatibilist" position, but Sapolsky's "Determined" makes a robust case).

And even if quantum physics are essential to human brains, it's not clear that introducing dice transcends determinism into free will, as opposed to simply adding unpredictability. ("The physics made me do it" -> "the dice made me do it") Let's not forget, there's a significant amount of randomness in AIs as well ("temperature"). And sure, it's simulated algorithmic randomness; but does that imply, if we somehow wired every `rand()` call into radioactive atom decay, that means the AI "wakes up"?

And all that is besides the point: I don't see any reason to assume "intelligence" requires "free will". I'm entirely capable of conceiving, in the abstract, a being which is both intelligent, and deterministic. You still haven't given me a definition (or better yet, a test), to distinguish a "real intelligence" from unintelligent mechanism.

reply
And quantum mechanics are what other than random probabilities?
reply