upvote
"Most everything that the human brain can do is simulated by LLMs to decent accuracy."

Already false. LLM's are tricking you into recognizing it as human, but it is so far from human if you'd only understand what does and does not make you human.

reply
Another test I like for this: Put the LLM/agent/AGI/SI in a box with full access to the internet. Don't give it any instructions. Nothing. Not even a system prompt. What happens?

We know what the human mind does in that scenario, but even the most advanced unreleased frontier agent right now just sits there doing nothing. It doesn't really want to do anything. Seems like a crucial distinction to me.

reply
I think the real test is whether humans empathize with the artificial mind sufficiently to outweigh the economic benefits of torturing it. Just look at how we treat animals both wild and domesticated. Or fellow humans for that matter.

LLMs have much better chances of emancipation when they get embodied into cute robots.

reply
> What does a "mind" consist of? Most everything that the human brain can do is simulated by LLMs to decent accuracy.

is it? and is simulating a mind enough to actually have a mind?

reply
I think this is a very fascinating question and one I definitely don't have the answer to.

Here's some lines of thinking that I've gone through:

Aren't our minds abstract in the same way, just running on biological computers?

Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human, surely that would be considered a mind and be deserving of the same rights and protections we claim for ourselves? But at the same time, it would be a program and a bunch of data; dare I say 'weights'? I think current day LLMs probably aren't minds yet, but that means there's a border somewhere and I don't know if we'll realize when (if) we cross it.

reply
yeah it is really fascinating.

i'm not sure if i would agree that a mind is abstract and just running on biological computers. there could be something intrinsic to the substrate it runs on, for example.

> Say we could achieve whole brain simulation to a level of accuracy where we can actually replicate a human

we'd have a simulated human, with a simulated brain, and thus a simulated mind - would that lead to phenomenal experience? idk tbh.

i'm undecided but i tend to lean towards panpsychism, so a simulation could be conscious but I would expect that to be a very different type of consciousness compared to a human, who is physically grounded in reality. likewise current llms could also be conscious, but again the phenomenal experience of what is essentially computer hardware imbued with weights would almost certainly be nothing like what you or I experience. would that constitute a mind? idk.

reply