> Weights helped me draft and proof this story.
Any HN reader here now, I encourage you to read the original ( https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s... ) in one sitting, go about your day, then read it again. Maybe make some notes on personal critical questions.
Now read the post's topic again ( https://maxleiter.com/blog/weights ) and reflect on the prior fact that weights helped [the author] draft and proof this story.
My reaction (and I'm sorry that it is harsh according to some) is that there is no intelligence found in either the author nor their tool. This is extreme navel gazing, based in science fiction, wanting (wishing) to believe those stories to be true.
I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.
The actual counterpoint is demonstrated in _Blindsight_ Peter Watts. He makes a strong (and rather terrifyingly strong) point that intelligence is not consciousness.
My original comment (roughly "there's no intelligence in this article, nor sentience in LLMs") is in response to the blog post's buried lede (that the cumulative activity of LLMs has accrued to a weight of "AGI is around the corner" or "there is artificial consciousness in this matrix").
To be clear, I'm not saying LLMs are useless or a wrong direction in development of "AI," but rather it's the Fool's Gold for the path towards AGI, the pursuit of the academic field of Artificial Intelligence research. A research that I've been abreast of for years before this new age of language models that has made everyone with a keyboard an arm chair expert.
Also, thank you for the book recommendation, it's on my list! :)
So I still disagree with your elucidated point (as you end with "which is valid"): the OP author is using prior art fiction to bolster their opinion of LLM-based software tools as being a possible vector of sentience, not to disarm our chauvinism like the original author intended. If OP wanted to make that point, they could have written a critical essay instead of farming out their thoughts as tokens.
But still, I look forward to reading the book you suggested to understand and appreciate your perspective more.
Blindsight is a remarkable book - I hope you enjoy it!
Plainly, based on the current ground trodded and the trajectory laid out by the frontier AI labs, I do not have concrete evidence/proof of sentience having emerged from LLM-based software tools as of June 4 2026 nor do I expect it to happen in the future based on my understanding and observations of this technology. I'm not excluding the possibility but wielding skepticism. I am open to being proven wrong with new discoveries.
Which is why (to return to my lashing of the dead horse) I don't see OP's post as worthwhile. Their post reiterates a point that is already valid (the prior art) with no new substantial discovery. Which is why "unoriginal and pointless" is apt, a novel idea was not presented; it's just some vain virtue signaling.
Also I don't see why intelligence not being consciousness is scary? My cats are very conscious as far as I can tell, but not particularly intelligent. I think LLM's exhibit some contextual intelligence without there being any particular reason to believe they're conscious other than woo psuedoscience.
That said, I don’t think it is useful for philosophy nor science to consider intelligence to be the same thing as consciousness. In fact I would go even further and claim that intelligence is not a useful construct, neither for philosophy nor for science. Consciousness, on the other hand, I think is useful for philosophy, but not (as of now) for science.
Thanks for saying this! It amazes me to witness so much pushback (in HN of all places!) for the call for skepticism and scientific rigor on claims made by business which have vested interests in hyping things up.
My personal theory is a fuzzy thought about how people want to reject the concept of a higher being and want to embrace the fact that we are now able to create our own consciousness and religion is dead.
I don't understand why, but it is the undertone of every argument I've seen that is pro-AI-is-sentient, like some big unspoken elephant-in-the-room.
I would rather just judge this tech on its own merits.
edit: this comment got 1 upvote literally as I submitted it. I know @ doesn't work, but @dang, something seems very strange about that.
There are many people who will categorically rule out the posibility of AI consciousness due to near-unshakable belief in a higher being. This argument resembles "Christians should not be worried about our climate since God is ultimately in control." Such views make it harder to collectively prevent dangers from a sentient AI, or harm to a sentient AI.
I do not claim that everyone who believes in a higher power believes concious AI to be impossible, or vice versa; just that it would be very hard to change the minds of those who adhere to this reasoning.
It's funny, because I find myself constantly stating the inverse of this. Every argument I've seen against AI being sentient plainly comes from, as you so eloquently put it, exactly "a place of emotion couched in logic". People desperately want it to not be true and will not take the logical step back of examining its actual similarities to human intelligence. Every argument comes down to "but it's not actually a human", or some variation of it -- which, if you pay attention, is not actually a logical counterargument. (Or, ironically, "but it doesn't have a soul", which is why the Pope is the perfect figurehead for these people).
If you already know any logical argument against it can be countered with "well doesn't a human brain work like that?", why are you so confident that your position is actually the logical one?
...And could it simply be that, alternatively, the concept is not actually a logical distinction, but rather an emotional one, made by emotional beings to put a word to what they claim makes them special?
That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true.
"Howdy-doodly-doo! Anybody like any toast?"
I am doing my best to communicate with you but to be honest you are not hearing me (across both responses), and I am out of words.
The toaster has hard coded or configurable weights. It makes a product from heat and time.
You could make toast by heating a brick on a campfire. It would be a clear sign of intelligence.
If we lift one weight out of your brain we wouldn't look at the number and say it is intelligence. It must exist in a chain or matrix multiplication to qualify.
The fun thing is that the timer and feedback mechanism do exist in a chain of events.
It's part of the system and it takes all steps to complete the task.
Our chain of thought that leads to making toast would be abandoned if it wasn't in working order.
It's difficult to tell who's trolling -- probably best to go with the charitable assumption that everyone is honestly trying to convey their opinion, but mostly talking past each other. Unfortunately these discussions about the nature of consciousness never go anywhere useful.
I think I'm probably in the same boat as you, roughly: a) LLMs are doing something really interesting that resembles in many ways both intelligence and consciousness; b) I suspect they're not actually conscious but I don't know how you'd know for sure; c) it all just drives home that we still don't really know what consciousness actually is. But like (a), it's definitely something really interesting...
"They're made out of weights" works precisely because LLMs really do have this mysterious property that they seem somehow intelligent even though nobody can explain exactly why, and there's active debate over whether they could be considered conscious.
The thing being discussed isn't simply an arbitrary MacGuffin; in both cases the nature of the thing is central to the impact of the story.
"Imagine how other intelligences would view us", written by us, hits a lot less hard when it's "imagine how our intelligences view a thing we are claiming is intelligent", not written by it.
This is well put. We don't need to imagine how a human views a llm because we can ... just do that. Everyone capable of reading the story is also capable of thinking about how they feel abouy llms that exist right now and you've probably used.
The trick of the original story is inverting your perspective, moving your view point fron yourself to an "other" (which I think is a primary qualifier for most good fiction).
Teapots are not compelling.
> You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.
God is compelling t billions of people.
Is Russel’s Teapot a bad argument in the God debate?
What's the relevance? If the argument made here are was a good argument, it wouldn't matter if Russell's argument was bad. We could construct a bad argument using reductio ad absurdum right here and now and it wouldn't matter to either argument.
Can you be straight with me? You know the salient difference between asserting the consciousness of a toaster and the consciousness of an AI, right? It isn't a mystery to you why we would find one line of inquiry interesting and the other not so much?
For instance, it's probably a real possibility in your mind that I am not a human and am an AI. But you probably aren't entertaining the hypothesis that I'm a toaster.
It directly parallels your argument.
> Can you be straight with me? You know the salient difference between asserting the consciousness of a toaster and the consciousness of an AI, right? It isn't a mystery to you why we would find one line of inquiry interesting and the other not so much?
There are two aspects here.
1. That people find the question interesting
2. That it has any bearing on reality (ontology?)
The first aspect is anthropology. Russel’s Teapot is not supposed to undercut any anthropological arguments. It’s supposed to undercut the second aspect.
So far you have said that the argument is compelling. What’s that got to do with reality? A robot cow could be sexually arousing to a real bull.
> For instance, it's probably a real possibility in your mind that I am not a human and am an AI. But you probably aren't entertaining the hypothesis that I'm a toaster.
Yeah. AIs know how to use computers. What’s this got to do with consciousness? Whether or not you are an AI is practical and disprovable. Consciousness is so ephemeral (for lack of a better word, not literally) that Philosophical Zombies is a real argument/thought experiment.
You may think I’m being coy (“Can you be straight with me”) but that’s not my intent at all.
Much like Russel argued that the burden of proof of God's existence is on theists, the burden to establish this parallel is on you as the person forwarding the argument. I don't see any relevant connection. Russel isn't arguing that a teapot is as real as God in the same way it's disputed here that a toaster is as conscious as an LLM.
> So far you have said that the argument is compelling. What’s that got to do with reality? A robot cow could be sexually arousing to a real bull.
AI is a real phenomenon that we can study and measure. There is no experiment that anyone has devised can determine whether or not they are conscious, so that is the reality - uncertainty. That doesn't mean they're conscious. It means that the belief they are not conscious is assumption.
You might say the same of a toaster, but these hypothesis are not equally strong. The toaster doesn't exhibit any behaviors to suggest that it's conscious. Consciousness isn't a hypothesis with any explanatory power for the observed behaviors of a toaster. It's not a hypothesis that's on the table. That's why the analogy doesn't work.
To put a fine point on it, it appears on it's face that AIs could be conscious. They can put on a very convincing performance of being a person. A sufficiently convincing performance is indistinguishable from the real thing. So at face value, the burden of proof is on them not being conscious. Reductionist arguments that present the mechanics of how they work and leap to their not being conscious don't work, because there is no law saying a statistical model can't be conscious. That's an assumption, not knowledge.
I pointed out the parallel in both statements. I can't do more than that.
> Russel isn't arguing that a teapot is as real as God in the same way it's disputed here that a toaster is as conscious as an LLM.
The teapot isn't real and the toaster consciousness is not real. What am I missing?
> AI is a real phenomenon that we can study and measure.
Robot cows are real as well.
> There is no experiment that anyone has devised can determine whether or not they are conscious, so that is the reality - uncertainty. That doesn't mean they're conscious. It means that the belief they are not conscious is assumption.
Yeah. You can't prove it for any entity. I agree.
> You might say the same of a toaster, but these hypothesis are not equally strong. The toaster doesn't exhibit any behaviors to suggest that it's conscious. Consciousness isn't a hypothesis with any explanatory power for the observed behaviors of a toaster. It's not a hypothesis that's on the table. That's why the analogy doesn't work.
The bull swears that the robot cow is a real cow. But we know better.
> To put a fine point on it, it appears on it's face that AIs could be conscious.
It doesn't to me. Not any facelength.
> They can put on a very convincing performance of being a person. A sufficiently convincing performance is indistinguishable from the real thing.
Objective reality has never cared (am I anthropomorphizing now?) what is indistinguishable for people.
> So at face value, the burden of proof is on them not being conscious.
Which party is the burden of proof on? This is confusing since you are saying that the burden of proof is on a position (on them not being conscious).
Is the burden of proof on people who argue that they are n o t conscious? That's peculiar.
I have never heard about any principle in philosophy or in science that says that, given enough Looks Like A Duck points, it is a duck. Based on subjective experience, even.
We obviously can't demand a falsifiable theory here. But we have to do better than arguing from incredulity.
> Reductionist arguments that present the mechanics of how they work and leap to their not being conscious don't work, because there is no law saying a statistical model can't be conscious. That's an assumption, not knowledge.
They don't have to rise to the level of disproving something for which they have no burden to disprove.
> I have never heard about any principle in philosophy or in science that says that, given enough Looks Like A Duck points, it is a duck. Based on subjective experience, even.
Also, you say this like "duck" isn't an arbitrary, artificial category created by humans, a "map not the reality" if you will.
There's very few things that humans can understand to the level of putting them into truly objective scientific categories (various pure elements maybe?), everything else we more or less bodge together for the sake of getting on with life.
It's not like conscious has some kind of formal objective provable definition, even inside the world of human created language and terms.
As far as I know, in the real world, if something looks enough like a duck (and can breed with a duck maybe) we, humans, do call it a duck.
You mean these statements?
No one cares about teapots in space either (Russel).
Teapots are not compelling.
> Which party is the burden of proof on? This is confusing since you are saying that the burden of proof is on a position (on them not being conscious).
It's metonymy. "X" stands in for "people who argue for X". May I ask if you were sincerely confused? You told me you aren't being coy but I have a hard time believing that this was so unclear.
> I have never heard about any principle in philosophy or in science that says that, given enough Looks Like A Duck points, it is a duck. Based on subjective experience, even.
If it looks like a duck, if it quacks like a duck, you should adjust your priors to assign a higher likelihood that it is a duck. All measurements contain error; you can't ever observe, "that is a duck," only "that looks like a duck". All knowledge is founded on a sufficiently deep stack of "looking like a duck" that we may assert it with confidence.
To the extent we have objective measures (like conducting a Turing test on blind participants), it can meet them too. You can't say the same of a toaster.
> We obviously can't demand a falsifiable theory here. But we have to do better than arguing from incredulity.
"It is a statistical model, ergo it is not conscious" is also an argument from incredulity. I don't know if that's your view or not but it's the one my remarks have been addressing in general.
Despite all the evidence that we are in fact just biological machines, people still persist the theory of our own uniqueness from other creatures, which we ourselves often treat as biological machines.
This adaptation is wonderful to me specifically because I think it shows that our shifted goalposts of, "well we're not just animals, we can think and reason" was never more than a convenient excuse for many people (and as evidence of animal intelligence continues to mount, denialism still attempts to preserve this distinction by claiming human thought and reason is different than 'animal' thought and reason, sans evidence).
It's not about who created it or why, it's about how people still haven't actually internalized the point, because the subject changing from human to LLM doesn't intrinsically change the message about consciousness, but the reaction being a 180 shows how hostile people are to that message, still.
And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach.
>You could perform exactly the same rhetorical trick with a toaster or anything else.
Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it).
P.S. Read the original too. Seems like the exact same could have been written about us instead of the original, if the focus wasn't on our substrate, but on our brain processing. Which, after all, is also about weights.
Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent.
For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.
I can even write one about a ruler, if I can bend it enough, no pun intended.
Doesn't that miss the whole point?
You could write "They're made of metal strips!". You wouldn't be able to write much else, as toasters don't have showcase in the way of human-level intelligent behavior. Which is the whole point in the meat and weights versions.
At best you could write "They're made of metal strips!" for toasters AND other metallic devices, and use some analogies of features BOTH have in common. But they wouldn't be intelligence related behaviors.
They can even adapt to their environment and the characteristics of the bread even with simplest of mechanisms because the text will be overglossing the fact that different types of breads have different thermal characteristics and this will deeply affect the behavior of the metal strips, bordering near a sentient being even more thoughtful and considerate than a human which is rushing through house to catch the bus in the morning.
Yeah, that's called "stretching it beyond any recognition".
You could do that. It will have none of the effectiveness or resonance of the two stories.
If you can't identify anyone, then this analogy doesn't work.
More than one, for many classes of devices, incl. toasters. Some were drunk, some were insane, and some were delusional.
LLMs are no different. They are automata, yet delusional people bring out pitchforks and torches when someone points out that they are just statistical models, and they don't even work when there's no input to them.
Which is very different than consciousness.
It would appear to me you have no interest in a real, good faith discussion on this topic because you think anyone who disagrees with you is necessarily delusional. Which is a shame, and that's the kind of dogma you are criticizing.
This was exactly the point of the story, it's too uncomfortable to admit that we don't know what consciousness is and what is and isn't conscious, so we just brush it under the rug.
My perspective comes from a set of pillars. First, I work at an HPC center, where we support running and development of AI systems, incl. international projects. IOW, I have knowledge about how these systems built, work and needs to continue working.
Moreover, I'm an HPC programmer myself, so I'm not completely uninformed about the math this involving this thing, and I'm lucky enough to have friends who are much more dedicated than me, and we discuss how this thing works and feels like this way.
I'm not an AI hater per se, being programmed AI systems in the past, incl. emergent intelligence systems with multi-agents which can span continents if need be (this was my master's thesis, time flies).
However, knowing what these things are capable of and how they are built. I don't believe them they're conscious/sentient beings. I also had much more time to ponder on these things even before LLMs being a thing. Some hard sci-fi books have asked these questions seriously in their captive adventures way earlier. If one reads these books seriously, there are a lot of philosophical angles to consider and draw upon.
I can discuss in good faith. For hours, days or months even, but throwing "you're a narrow-minded dogmatic luddite neanderthal!" card to anyone disagreeing with you is not it.
It's perfectly fine to believe they are not conscious, I am not convinced they are, but asserting anyone who disagrees with you is delusional is unfortunate.
No, I didn't assert anything. I have just given examples rooted in my experience.
None of my friends who also happen to know how these things work told or defended that they are conscious, even intelligent. Maybe my friends are dumb, I dunno.
Once I have seen a man who claimed that evil has possessed the POS device at his desk. The thing was printing "cannot connect to server" on the receipt printer every 10 minutes, yet he didn't know how that thing worked, and was a bit too high to read the paper the thing was printing out.
This age's LLM craze is akin to "wonder inventions" of 70s, which are deemed dangerous or harmful in the future. LLMs will be with us, but we need to pass beyond the hype and stop sweeping the problems they create (environmental and societal) under the proverbial rug.
Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.
I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the improbability of something like thinking, feeling, etc arising from a lowly substrate.
And having read both now, I see how the existence of the original is a plus for this story, not a minus. Instead of making look like mere copy (as would be the case for a typical story modelled after another), in this case, it adds a meta layer, and enriches it.
You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.
What I think a lot of people are getting stuck on is that editing the text to say 'toaster' would not mean that toasters were conscious, and that editing the text to say 'weights' doesn't mean that weights are conscious either. Stories aren't factual just because they are written.
The original story was written by the thing claiming to be conscious; the LLM and toaster ones would not be, which undermines the claim to consciousness a lot.
The story does not assert that search and replacing "meat" with "weight" makes them conscious through some magical mechanism. It's a thought experiment.
You're falling into the trap of assuming that a good presentation--being convincing--is the same as being truthful.
Yes, there very much is a huge barrier. The copy and the original both keep the same subject matter: intelligence/human-like behavior.
The toaster doesn't. We could do edits, and the story (original or copy) would lose all its potency.
The surprise is "but how is the richness of intelligent behavior produced from something as basic as meat/weights". Why is kind of surprising reductionism is both cases.
Whereas nobody is surprised that "a metal strip with electricity flowing through it from a power source" heats pieces of bread to a specific temperature. Even if a toaster was "metal strips" all the way down, it's nowhere near as impressive jump from substrate to behavior, nor is the behavior as important to us and touches the core of our existance.
That is entirely separate to whether or not it's meaningful to write a story about a conscious toaster. Again, expressing an idea in fiction is not related to the accuracy of that idea.
What makes the original story interesting is that it was written by the thing claiming to be conscious, which is what takes it from 'a story' to 'a story making an important point'. That's not the case with a hypothetical story about a toaster, and it's not the case with this story about LLMs.
The story is convincing because it's well-written, not because it is factual.
And you're misunderstanding me. There is no practical barrier to writing this story about a toaster. But there is a conceptual one. It would be bullshit, that has little resonance, and little connection, as the whole point of the meat and weights stories is the "lowly substrate -> surprisingly intelligent human (or human-like in the case of the weights version) behavior".
Nobody thinks a toaster's behavior as surprising or intelligent or human-like to begin with.
So you'll just be stretching the analogy beyond all recognition, with little to no payoff.
>Writing a story in which a toaster in conscious is not even new
It's not, but it has nothing to do with the meats or weights stories' point, as it's just an arbitrary choice, like making the candles and teapot sentient in Beauty and the Beast.
Whereas in the meat and weights stories the whole point is the surprise from us already seeing the human or human-like sentience of the thing, and comparing it to the "dead" substrate.
Fixed the link for you: https://www.youtube.com/watch?v=LRq_SAuQDec
(Hah. It gets eerily relevant starting at 2:37)
It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were.
But it doesn't really give a reason to accept the analogy, it just asserts it.
There's a big difference between a whole civilization and a piece of software that can output text.
It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.
>"A side effect. You're asking me to believe in sentient weights."
Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.
The original:
>That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat.
I think it was adapted by an LLM which didn't quite get the meaning.
Extending the idea to a different context based on new material conditions is as human as it gets really.
This did not add anything, just rephrased it so rather than humans viewed through the pov of aliens it's LLMs viewed through the pov of humans. Well, we are the humans, so surely we do not need to learn about this point of view?
It is not. That is why the "trick" can be performed on anything. That is the point. To show you that consciousness only needs to be a set of weights, not that far off from a toaster. It trivializes what it is to be human and it's also extremely true. Consciousness is a trivial thing, we make it out to be big/important/profound and that is the delusion.