https://www.edge.org/3rd_culture/ramachandran07/ramachandran...
In short, as far as I can remember: evolutionary, it makes sense to understand other humans, to feel what they feel(empathy - the mirror neurons system), and simulate their thinking and feelings.
And once we have those systems, we can also use those on ourselves. And that's consciousness.
Edit:And I wonder if this is a testable hypothesis, in a simulation.
For reasons like that, I don’t think that P-zombies are possible.
(See https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zo... for a much longer version of this argument)
Think about it from an evolutionary perspective:
Animals that step into a lava flow or forest fire don't reproduce. Eventually some evolve the ability to detect intense heat from a distance, and pain as soon as tissue destruction is imminent. They do not have nor need a general understanding of the dangers of heat, or even conscious awareness that they've stepped on a hot coal.
The pain signal compels them, but that is very low level machinery. It had to continue compelling beings that developed larger and more sophisticated brains that are capable of abstract thought and reasoning. Feeling pain is one of the lowest level parts of the brain telling the higher parts exactly what its going to do in terms that permit no disagreement.
Not sure about taking it down to the level of consciousness, but makes sense regarding the sense of self, the conceptual experiencer, the perceived center of experience. It agrees well with the observation I have made again and again they my sense of self is much stronger when I'm around people, and stronger still when I'm in a context where I don't know people and/or am uncertain in social rules.
This can be as immediate as dancing in a club, and closing my eyes I feel open, free, still, the body just flowing, then opening my eyes and feeling the cage of categorization of the world, relating my self to other people as a major function, coming right back.
Also being alone in nature for me makes the sense of self drop. Without intention, spending even just a few hours alone in a forest seems to quiet down the part modeling my self in relation to the world so much. There's no need for it there. I'm not a person in a forest; I become the trees, the birds, the rustling of the leaves, the sun shining through the canopy.
I know that the part of the brain responsible for the self thoughts is called the "default mode network". and meditation can reduce it's activity, i.e. the internal monologue stops, but also it can be measured via FMRI.
So i wondered: are the mirror neurons part of the "default mode network"? I asked claude that, he said no, they are two different systems.
So maybe the mirror neurons, those responsible for empathy, "to feel as someone else" are also responsible for becoming the trees, the birds and the rushing of the leaves?
1. Simulate others' thinking and feelings.
2. ???
3. Consciousness!
Why would one lead to the other? How would one lead to the other?
It's terrific, but the poetry is from the original it links to, in case you didn't realise.
It's a brilliant and timely update though.
Aside, there are various recorded versions including video on YouTube but this is my favourite, a radio play:
They're Made Out of Meat
https://www.wnycstudios.org/podcasts/studio/segments/168264-...
The self-modeling, is in such a tight loop, it melds "ourselves" and our model of ourselves, our thinking and choices, and experience of our thinking and choices, into one component.
Like you can't analyze half a wheel of a bicycle and be talking about the same thing.
This awareness, increased modeling, control, feedback loop has tightened up over many stages. Just a few:
1. The body-sense loop
2. The internalized-environment-model loop
3. The body-internal-function loop
4. The body-internal-model loop
5. The emotional-cognitive loop
6. And finally, the tightest loop of all, our high-level cognitive activity, experienced as feedback directly, our self-model, and our self-direction, all merged into one thing.
We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.
That is consciousness. Rich self awareness, a merger of self-model and self-direction, and all in service of understanding and managing ourselves. Hw we can leverage our greatest tool, our self-directable mind, its habits, views, and behavior.
This wasn't an accident. A happy side-effect of our brains. It is a biologically evolved focusing of our highest-level behavior, with tight feedback, constant self-modeling and continuous focus on our inner status as motivation and most privileged object of our control. It has been ruthlessly optimized for, for a very long time.
> That is consciousness.
So thinking is consciousness?
Can there be consciousness without content? E.g. can I just be conscious of being conscious? If so, consciousness cannot be defined as the thing(s) we're conscious of.
Being conscious of being conscious means that there is content. You are conscious of something.
It’s a bit like a Gödel statement that quotes itself, that is a statement about itself. It doesn’t mean that it has no content.
Thinking isn’t consciousness. Consciousness doesn’t require thinking, it only requires perception. The perception of a process of perception within the same mind might constitute consciousness.
Constant self-awareness, self-experience, self-focus, self-management, and self-improvement of one's own self (mind), is going to be an adaptive behavior for anything intelligent with resources to leverage. Whether truly independent, or highly motivated to serve others. The mind is the greatest tool.
I think that is more than simply a good functional definition of consciousness. How could all that integration and self-integration not be conscious.
You would really like Michael Pollan's latest book [1], entirely devoted to his exploration of consciousness researchers' POVs on this exact topic.
My favorite quote is that ~"perhaps Descartes was only half-wrong when suggesting I think, therefore I am; it seems rather closer to I FEEL, therefore I am."~
[1] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>
----
I've grown thousands of plants; I've read two of the author's other books devoted to plants; in this book Pollan makes compelling arguments for plant sentience (over a much-longer timeframe).
Sure, perhaps plant consciousness is a bit of a stretch, but they're certainly intelligent and curious creatures. He makes both arguments supporting plant volition.
----
If you haven't seen My Octopus Teacher (Netflix), do. I'm a bald 275lb bluecollarguy... and I wept/awed (both). So beautiful, we bundled neurons.
----
Bonus quote ~"color is where reality and magic appear as-if together"~ [color isn't real, but is perceived]. We most-often see what's most-predictable, not necessarily what we actually detected [in the case of color: nothing but nanometers].
What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.
I would think we are so far away that there will be unknown unknowns we encounter on the way.
Yeah, but it's hard to explain this to people, especially AI-pro people. Too many are convinced that all we are doing is a cut-down version of the human brain, and it's hard to explain to them that, no actually, we aren't modeling the human brain to the level of granularity you think we are.
https://www.quantamagazine.org/ai-is-nothing-like-a-brain-an... https://pmc.ncbi.nlm.nih.gov/articles/PMC9665914/
This is why making more neuromorphic NNs is still an active area of research, although they typically all focus on another extremely simplified model (spiking neural networks).
And it seems that, given enough input/outputs/compute, it is possible to train the necessary function.
Details of how the building bricks look like (matmul, electromagnetism or quantum effects) are not that relevant in the broader picture.
What is missing right now, is the fact that the function in question changes over time in biomachines, while our LLMs are static at inference time.
a) The brain might have an entropy source (then it can't be modeled as a function). Trivially to fix, and in some sense, with diffusion models starting from random numbers, AI has done so.
b) The hidden function might be not computable. I would have no idea how that would work, but I think this is what it boils down to if people say "the human brain is more than a machine".
b) well, it can be the case that, say, certain kinds of computation are either too inefficient or outright impossible within the current model.
Who knows...
If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.
This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).
LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.
I'm not entirely convinced by that second paragraph, but I think the logic holds together.
I think that consciousness comes before self-awareness, even though self-awareness is kind of a vague term. Self-awareness can either be an abstract knowledge that you are an organism and a discrete entity in the world (world knowledge/self knowledge), or it can be more basic and be a form of conscious experience, but as my point was, I think conscious experience is broader and does not necessarily need to be about self-awareness.
Some questions are just ill formed.
Plus even if "LLMs are alive and conscious", this still would scratch the surface of the morale/ethical/societal considerations that people really care about.
Because even with other humans, we can argue if they exist or if they are mere npc in a solipsist world view.
We say "it's raining" but that doesn't imply agency to whatever is causing it to rain.
Also, please don't invoke solipsism, if you want to debate that, you're by definition obligated to do it with yourself.
About solipsism, yes sure.
I didn’t had time to add it at the moment, but I initially also wanted to discuss more on the prosaic points. So here we go:
Recognizing sentience/consciousness to something actually is not that a big factor in defining behavioral response. Let’s just look at how humans are treating other humans, other mammals, other animals, other life forms. Or taken the other cultural scheme around, in some animist folk one can perfectly consider that any rock on the ground have really an inner soul life just as much as oneself, it doesn’t mean it will consider possible interactions with the rock as equivalent. And, wink to the Overton window, practicing institutional cannibalism doesn’t imply that eaten people are disrespected in their dignity, quite the contrary.
All that to say that "are LLMs conscious" is not even a moral important point. Paperclip maximizer² and can the famous Dijksta’s quote "The question whether computers can think is not more interesting than the question whether submarines can swim." already show how pointless morale debate on LLM and consciousness are, even when one consider potential logistical threats/opportunities they could represent.
¹ https://english.stackexchange.com/questions/123739/its-raini...
² https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape...
https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...
I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own.
Consciousness itself doesn't have a well agreed upon definition, but I would posit that _most_ people would agree humans have it, and _most_ people would agree individual cells (neurons) do not have it. If you agree with those two statements, then consciousness is an emergent property by the definition I gave above
We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.
Why cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?
Previous comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.
When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.
However there is also the ethical issue of creating a consciousness and then destroying/murdering it.
The logical conclusion is that the brain makes me.
Consciousness can be not-emergent but also not metaphysical, think sci-fi-type undiscovered physics or matter.
Of course both of those suffer from the recursive problem of just kicking the can one level up. But I guess that's fundamentally unsolvable so who cares.
One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.
Why is that?
But I think my issue with the emergence theory is that it seems to imply to me that consciousness is non-physical and non-local. So what entity is actually experiencing the consciousness? It's not that I believe consciousness is physical and local, but people who make the emergence argument seem to believe it is and I can't figure out how that is supposed to work.
A linear dynamic, say x_n+1=lambda x_n, or x_n+1=(1-x_n), is never chaotic. But if you multiply them, x_n+1=lambda x_n (1-x_n), it depends on lambda if the system is chaotic.
None of the components are chaotic. But for specific combinations, chaos emerges as an property.
In physics, the mass of mesons and the nucleon is emergent. It's completely different from the constituents' mass. Different from an atom, where its mass is very close to the mass of its nucleus and its electrons.
Or something like that. This gets to the "dorm room bullshitting" level right quick.
Idk, it's really hard to articulate my thoughts here and yes it is pretty close to the conversations I had in college on various substances. Lol.
And we are only doing it for a few decades. Evolution had million of years of "try and error".
Psychological time is your own weights being updated in response to stimuli and internal processing.
When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.
Although part of me thinks some of this is from being substantially busier than ever (work + kids), and hoping maybe it can slow down again, at least a little bit.
If you'll allow me to interpret "speak" to include "understand", I will respectfully add a contradictory note. My dog has a vocabulary of at least dozens of words and understands them remarkably well. For example, different areas we can go to have different names and saying one of them gets her to make an immediate hard turn.
I would also argue that dogs have a gesture and body posture based language they use among themselves. They, like most other animals, are not able to make the variety of noises we humans make, so they use movement instead.
I personally can easily believe that self-awareness/consciousness and language are both near-unavoidable side effects of emergent complexity, and exist in degrees across nature.
Here's a more general idea. Our modern physics says that the whole universe is filled with fields and field is composed of numbers. What if we take that literally? When we say an electron is present here, we actually mean that there are more copies of particular number superposed at that place.
Yeah, the weights not updating online makes them less like a living organism that can update and learn and evolve ... ok ....
I could say something like "the reason why people and animals are conscious is quantum mechanical effects". Ok, maybe, it would be hard to prove me wrong because nobody knows, but it's not a very useful statement by me if I can't tell you why you should believe me on that.
How do you actually know the Chinese room isn't conscious? It's merely obvious that it isn't, but that's not evidence.
(Spoiler, it was not about consciousness)
What is no longer conscious, the brain? Or the body? Or some other entity?
If consciousness is weakly emergent, how do we know it emerges from the solely from the brain and not, say, brain + body? Or brain + body + or environment. Or from the universe itself?
Then you shouldn't have dropped out of your linguistics programme.
Although it does amusingly do what annoys critics of AI: take something good written by a person, steal it and slop it up while overall misunderstanding it.
I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”.
Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly impossible with any amount of computing power…
Today I asked my employer’s Claude to proofread a short software user manual written in markdown. (Trying this with a LLM was a first for me!) It pointed out not only grammar mistakes but also cases where I did not follow my own self-imposed conventions that were never explicitly stated. (I didn’t have a chapter detailing all the typographical conventions the way specification documents often do)
I also asked it what parts might be unclear to a user. The response was surprisingly good — no worse than asking the QA tester for the same feedback.
Also find the LLM seems to “comprehend” subtle technical details of obscure technical specification documents that nobody on the Internet ever discusses.
As for time and the universe, Stephen Wolfram’s theories seem intriguing. He seems a bit obsessed with pretty diagrams but the idea of time dilation being the result of computation seems somewhat more appealing than trying to imagine relationships between time, gravity, and the speed of light .
Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.