upvote
I could in principle implement a spreadsheet or terminal emulator in human neurons, and we would agree that it isn't conscious. That has nothing to do with whether or not humans are conscious.

Clearly consciousness is an emergent property of certain kinds of network, independent of the substrate within which the network exists.

reply
> Clearly consciousness is an emergent property of certain kinds of network

I happen to agree that this is likely, but it absolutely is not "clear" that this is the case.

reply
Yeah but surely it is though, obviously.
reply
Agreed. Some have even trained neural networks made of actual biological neurons to play doom[0]. Brain cells! Doing smart things!

I still wouldn’t argue that this brain in a Petri dish is, in any way, conscious. Despite it sharing the exact same substrate as everyone around me.

[0]: https://www.theguardian.com/games/2026/mar/16/petri-dish-bra...

reply
If you could do that practically and in reality, and get back to us with the results so we can debate them, that would be great.
reply
http://news.bbc.co.uk/2/hi/sci/tech/358822.stm

Leech neurons used to implement a calculator.

Are you suggesting that applications above some level of complexity can't be implemented using biological components? Because I'm pretty sure all I need to show you is a NAND gate in order to prove that arbitrary computation is possible.

reply
Aren’t, right now, not can’t.
reply
Can you explain why you think that matters?
reply
I’m enough of an empiricist to want to deal with reality it is. Thought experiments about things we “could” be doing or “could” implement and then already know the result of because we can rationalise out the consequences of the thought experiment don’t really interest me until they’re done in reality.

Like for example the China Brain - https://en.wikipedia.org/wiki/China_brain. We could in theory organise that and it should in theory be a functional representation of the brain and should in theory prove a functionalist metaphysic of consciousness, just this overlooks the fact that trying to organise that kind of experiment in our reality with a real 1.4 billion people is impractical to the point of impossibility, so in reality it proves nothing.

Or take the hoverboard from back to the future - this seems like a fairly plausible device and is easy to think about, and I could be writing speculative papers about our hoverboard future and what it means for transportation, but when it boils down to it implementing the thing in the way we all believe it should work, like a kind of gravitationally repulsive force field, doesn’t seem like it’s part of physics. I’d want to wait until the day until science delivered an actual good-enough hoverboard that works the same as the one in Back to the Future.

reply
Empiricism doesn't mean that you refuse to accept the existence of something until you witness it. It means that beliefs have at their root direct observations, which in conjunction with a logical argument support that belief.

According your definition of empiricism, you wouldn't accept that next years processors will any more powerful than this years until they've been built and you've run a benchmark, which would be a little dumb.

Instead, I'm assuming you combine your knowledge of scaling laws, the basic physics of computation, and your direct observation of the power of modern processors, and infer correctly that next years yet-to-exist machines will be more powerful than those available today.

Re the China brain: its a thought experiment designed to illustrate substrate independence, it is indeed unfortunate that we can't run the experiment. You could absolutely get a small group of individuals together and teach them to reproduce the behaviour of an individual neuron. That would then be enough to demonstrate that a whole brain is possible.

Re the hoverboard: anyone with a basic knowledge of physics knows that this is not possible. Since you have the capacity to reason and access to evidence, you don't have to wait, you can say confidently that it cannot happen unless our physical theories are profoundly wrong.

reply
Well I said I’m “enough of” an empiricist, not that I was denying what you lay out. But in the case of such thought experiments that can be begun with a small empirical circumstance such as a small group of individuals and then logically extended, a lot of the time I’d still want to say “I don’t believe that the logic is a simple as you suggest to extrapolate”. While I’m not in favour of radical skepticism, I find a healthy skepticism in the face experiments which could theoretically be run to test their validity - in this case the skepticism shows that “no, you can’t actually organise a China brain, ever, because we can’t corral 1.4 billion people or even get them on board with such an idea” so I’d find the logic faulty that says there’s a proof for functionalism regardless within the thought experiment.

A basic knowledge of today’s physics would say the hoverboard is not possible. An advanced knowledge of physics five-hundred years hence might show a way of doing it. It “wasn’t possible” to build a thinking, talking machine five hundred years ago and look at us now.

reply
I'm curious about what you do for a living, if you'd indulge me?
reply
It's not clear at all that consciousness is independent of the substrate. See the Harder Problem of Consciousness by Ned Block: https://www.jstor.org/stable/3655621
reply
It is clear that consciousness is independent of the substrate if you don't believe in magic.

We could make a very dumb biological calculator out of a few genetically-engineered neurons that would very obviously not be conscious.

It's still an open question if we can embed consciousness in our current microchips if we had enough of them together (which I think we currently don't), or if it requires some other physical process we don't fully understand, e.g. quantum. I strongly doubt it does require any quantum shenanigans, but even if it did, we can and will find all sorts of ways to make computers that can perform those shenanigans too. Eventually we're just going to stop being able to move the goalposts, unless you set those goalposts in magic-land.

reply
Quantum physics isn’t shenanigans, it’s completely fundamental and necessary to the brain and its electrochemistry. To say “yes but we can create a model of the brain which doesn’t need all that, we can model the classical physics or a reduced set of quantum interactions (running classically)” and then model the brains neurons on a computational substrate, well this is going to be nothing like the brain so won’t give you the result you’re looking for.

And say what you want about meat but we don’t seem to find consciousness in rocks or plants or clouds or hairdryers. And the buddists report that some very strange things happen if you meditate for years on end but obviously they must be talking shit and making it up because it’s not testably scientific.

reply
Are you familiar with these things called emulators?
reply
Totally agree. There's nothing magical about meat.
reply
Most spreadsheet engines are turing complete, so you could use them to run an LLM.

I don't think many people would say an LLM written in Python is conscious BUT an LLM written in Excel is not.

People just don't ascribe consciousness to things that can't converse (or at least emote or give the appearance of emoting), and spreadsheets don't do that.

The reason people are debating the consciousness of LLMs is - obviously - that the LLMs generate sufficiently plausible text that people using them think they're having a two-part conversation. Like I think I might be having a two-part conversation now. Turning your question around, why do you think Hacker News posters are conscious? You have no direct evidence they are.

reply
I think it's not really about having a conversation - I mean, that's part of it, but alone it's an illusion that eventually fades quickly. It's more of because of how it demonstrates intelligent behavior in reaction to requests, both in trivial and complex matter, and all across the board. LLM's response may be completely incorrect or confused, but it's nearly always exactly what you expect from a human[0]. This creates a more general feeling you're dealing with a human-like intelligence.

To be clear: I'm not talking about surface level things like prose. I'm saying that no matter what you do - whether you just paste a truncated log of a command into it with no further comment, or talk like a drunk teenager with no appreciation for grammar, or mix natural languages, or mix natural languages and JSON, or whatever else, the reaction you get is always that you would expect of a helpful person that got your message. It'll try - and usually succeed - to parse out what you actually meant, and deal well with subtleties around it.

This alone may not be enough to call it conscious or intelligent, but at the very least it's a large leap in that direction, and a qualitatively new functionality that classical software does not posses.

--

[0] - This is by design, not accident. "Respond to arbitrary input in a way that makes sense to humans" is literally the overall goal function the LLMs are trained to.

reply
Ok then, when my GPU runs No Man’s Sky, I don’t get confused and think it’s running a universe and that universe is real, nor that anything about that system is conscious. When I close the game and load the LLM, I still don’t think the same machine has a case for consciousness even though I think it’s super smart and helpful.
reply
It is difficult when humour and trolling are forbidden. It was easier to tell Slashdot posters were conscious. We could easily reach the stage soon where your agent is responding to my agent and we just leave them to it to run HN automatically.
reply
Why do you assume other humans or cats and dogs are conscious? I'd suggest it's because they behave as if they are. LLMs show some signs of human like awareness, at least of text contents, whereas spreadsheets don't.

When doctors are testing if humans are conscious they'll do things like hold out their hand and say how many fingers am I holding up. Some LLMs can pass that.

reply
I’d stay that my belief about other humans or cats or dogs wasn’t really an assumption based on some kind of proof that had to be constructed from a ground of doubting. I’d say it was the type of hinge certainty upon which my world depends and my doubting it would only ever arise for a few minutes in special circumstances. Same as I could ponder gravity turning off and bouncing around the room but this has never happened and seems like it never will, so isn’t really much of a doubt worthy activity, same as questioning my dog’s consciousness.
reply
I don't think LLMs are conscious but you have to admit that they are designed to have the __appearance__ of being conscious. Hence the debate!
reply
> Why do you have no inkling that your spreadsheet or terminal emulator is conscious, yet when that same machine is running an LLM all of a sudden we’re debating its consciousness?

Difference in size and complexity and nature of calculations being run?

I'd ask the other way - why do you (general you, people who do not have this inkling) have no problem debating consciousness of meat based brains, but it somehow becomes a category error when talking about silicon? Assuming you don't believe in divine magic, and that divine magic is core to consciousness, there's no reason to assume it's impossible a complex enough machine running complex enough software could be intelligent, or conscious - thinking is computation, and computation is made of math - it's independent of substrate that does the computing in the real world.

LLMs are definitely a different beast than regular software - both in their structure and in their generality. They may not be conscious or intelligent, maybe this specific design could never truly be (though I think it could) - but bucketing them with spreadsheets and terminal emulators is a real category error. If you stop fixating on the underlying substrate, then LLMs are already much more similar to biological minds than to any "regular" computer programs.

But that's still somewhat abstract. In immediate practical terms, it's also why I keep saying that anthropomorphising them gives a better high-order intuition: they are, by design, emulating human thinking in full generality, which makes their overall behavior, including their well-known problems like hallucinations or prompt injections (i.e. manipulation/gullibility), match what you'd expect of a people-like component of a system. It's a real, dangerous mistake, to treat LLMs like regular software components when designing systems.

reply
I wouldn’t say they’re anywhere near “full generality” because that would include believing untruths and passionate rages, existential angsts and the crazed obsession of heartbreak.

Your definitions depend on us being computers who think.

Solving some of the intelligence part of our logical thinking doesn’t get us anywhere near consciousness, which is a superset way beyond the linguistic intelligence used for communication.

reply
>Why do you have no inkling that your spreadsheet or terminal emulator is conscious

I personally believe all information processing machines possess some level of consciousness.

reply
PRAISE BE THE OMNISSIAH!
reply