Maybe we need to start thinking less about building tests for definitively calling an LLM AGI and instead deciding when we can't tell humans aren't LLMs for declaring AGI is here.
Isn't that exactly what you would expect to happen as we learn more about the nature and inner workings of intelligence and refine our expectations?
There's no reason to rest our case with the Turing test.
I hear the "shifting goalposts" riposte a lot, but then it would be very unexciting to freeze our ambitions.
At least in an academic sense, what LLMs aren't is just as interesting as what they are.
Does it matter?
We can do countless things people in the 90's would think was black magic.
If I showed the kid version of myself what I can do with Opus or Nano Banana or Seedance, let alone broadband and smartphones, I think I'd feel we were living in the Star Trek future. The fact that we can have "conversations" with AI is wild. That we can make movies and websites and games. It's incredible.
And there does not seem to be a limit yet.
The Turing Test/Imitation Game is not a good benchmark for AGI. It is a linguistics test only. Many chatbots even before LLMs can pass the Turing Test to a certain degree.
Regardless, the goalpost hasn't shifted. Replacing human workforce is the ultimate end goal. That's why there's investors. The investors are not pouring billions to pass the Turing Test.
AGI - Automatically Generating Income.
> I propose to consider the question, "Can machines think?" This should begin > with definitions of the meaning of the terms "machine" and "think." The > definitions might be framed so as to reflect so far as possible the normal use > of the words, but this attitude is dangerous, If the meaning of the words > "machine" and "think" are to be found by examining how they are commonly used > it is difficult to escape the conclusion that the meaning and the answer to the > question, "Can machines think?" is to be sought in a statistical survey such as > a Gallup poll. But this is absurd. Instead of attempting such a definition I > shall replace the question by another, which is closely related to it and is > expressed in relatively unambiguous words.
Many people who want to argue about AGI and its relation to the Turing test would do well to read Turing's own arguments.
Like do people not know what word "general" means? It means not limited to any subset of capabilities -- so that means it can teach itself to do anything that can be learned. Like start a business. AI today can't really learn from its experiences at all.
The truth is, we have had AGI for years now. We even have artificial super intelligence - we have software systems that are more intelligent than any human. Some humans might have an extremely narrow subject that they are more intelligent than any AI system, but the people on that list are vanishing small.
AI hasn't met sci-fi expectations, and that's a marketing opportunity. That's all it is.
also, I'm pretty sure some people will move goalposts further even then.
If you've never read the original paper [1] I recommend that you do so. We're long past the point of some human can't determine if X was done by man or machine.
Regarding shifting goalposts, you are suggesting the goalposts are being moved further away, but it's the exact opposite. The goalposts are being moved closer and closer. Someone from the 50s would have had the expectation that artificial intelligence ise something recognisable as essentially equivalent to human intelligence, just in a machine. Artificial intelligence in old sci-fi looked nothing like Claude Code. The definition has since been watered down again and again and again and again so that anything and everything a computer does is artificial intelligence. We might as well call a calculator AGI at this point.
An AGI would not have problems reading an analog clock. Or rather, it would not have a problem realizing it had a problem reading it, and would try to learn how to do it.
An AGI is not whatever (sophisticated) statistical model is hot this week.
Just my take.
LLMs aren't artificial superintelligence and might not reach that point, but refusing to call them AGI is absolutely moving the goalposts.
That's not the definition they have been using. The definition was "$100B in profits". That's less than the net income of Microsoft. It would be an interesting milestone, but certainly not "most of the jobs in an economy".
It ties the definition to economic value, which I think is the best definition that we can conjure given that AGI is otherwise highly subjective. Economically relevant work is dictated by markets, which I think is the best proxy we have for something so ambiguous.
And then I think coming up with the right metric is just as subjective on this field as the technological one.
Deep scientific discoveries are also cognitively demanding, but are not really valued (see the precarious work environment in academia).
Another point: a lot of work is rather valued in the first place because the work centers around being submissive/docile with regard to bullshit (see the phenomenon of bullshit jobs). You really know better, but you have to keep your mouth shut.
e.g. average cost to complete a set of representative tasks
Huh. Source? I mean, typical OpenAI bullshit, but would love to know how they defined it.
Wow. Maybe they spelled it out as aggregate gross income :P.
Apple, Alphabet, Amazon, NVIDIA, Samsung, Intel, Cisco, Pfizer, UnitedHealth , Procter & Gamble, Berkshire Hathaway, China Construction Bank, Wells Fargo, ...
A self-running massive corporation with no people that generates billions in profit, no matter what you call it, would completely upend all previous structural assumptions under capitalism
That's a relevent aspect of the AGI concept.
[0] https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
if you think drone targeting in Ukraine is scary now, wait until AGI is on it...
ditto for exploiting vulns via mythos
I responded to the below quoted question you dumb fuck. Can you figure out basic website navigation. Or is that too complex for you?
----- ' They redefined AGI to be an economical thing Huh. Source? I mean, typical OpenAI bullshit, but would love to know how they defined it. '
BTW I didn't downwote you (I hate it, if many people downvote a comment it's harder to read), I was just trying to explain why others did. On second thought, my comment was wrong, because your answer was related to the question but it wasn't really the intended one.
I don't think your original comment deserve to be downvoted. (Calling someone illiterate, on the other hand.)
But the "it" I was asking about was "AGI" as "an economical thing." You technically correctly answered how OpenAI defines AGI in public, i.e. with no reference to profits. But it did not address the economic definition OP initially alluded to.
For what it's worth, I could have been clearer in my ask.
But originally I was just trying to be helpful by quoting their charter on what they consider "agi" now.
"OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits."
Given that the definition of AGI is beyond meaningless, it is clear that the "I" in AGI stands for IPO.
[0] https://finance.yahoo.com/news/microsoft-openai-financial-de...
I don't get why HN commenters find this so hard to understand. I have a sense they are being deliberately obtuse because they resent OpenAI's success.
The current estimation on the time between this is fairly small, bottlenecked most likely by compute constraints, risk aversion, and need to implement safeguards. Metaculus puts it at about 32 months
https://www.metaculus.com/questions/4123/time-between-weak-a...
I don’t really buy into the ”one part equals another”, we are very quick to make those assumptions but they are usually far from the science fiction promised. Batteries and self driving cars comes to mind, and organic or otherwise crazy storage technologies, all ”very soon” for multiple decades.
It’s very possible that white collar jobs get automated to a large degree and we’ll be nowhere closer to AGI than we were in the 70’s, I would actually bet on that outcome being far more likely.
The leap between Opus 4.7/GPT 5.5 and what would be sufficient for AGI seems smaller than the leap between The invention of the Transformer model (2017) and today, thus by a very conservative estimate I think it will take no more time between then and now as it will between now and an AI model as smart as any human in all respects (so by 2035). I think it will be shorter though because the amount of money being put into improving and scaling AI models and systems is 100000x greater than it was in 2017.
From Wikipedia
Eschatology (/ˌɛskəˈtɒlədʒi/; from Ancient Greek ἔσχατος (éskhatos) 'last' and -logy) concerns expectations of the end of present age, human history, or the world itself.
I'm case anyone else is vocabulary skill checked like me
Russian Invasion - Salami Tactics | Yes Prime Minister
OpenAI and Microsoft do (did?) have a quantifiable definition of AGI, it’s just a stupid one that is hard to take seriously and get behind scientifically.
https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
> The two companies reportedly signed an agreement last year stating OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits. That’s far from the rigorous technical and philosophical definition of AGI many expect.
People obviously have really strong opinions on AI and the hype around investments into these companies but it feels like this is giving people a pass on really low quality discourse.
This source [1] from this time last year says even lab leaders most bullish estimate was 2027.
[1]. https://80000hours.org/2025/03/when-do-experts-expect-agi-to...
Why are we expecting AGI to one shot it? Can't we have an AGI that can fails occasionally to solve some math problem? Is the expectation of AGI to be all knowing?
By the way I agree that AGI is not around the corner or I am not arguing any of the llm s are "thinking machines". It's just I agree goal post or posts needs to be set well.
They can. If one consolidated the AI industry into a single monopoly, it would probably be profitable. That doesn't mean in its current state it can't succumb to ruionous competition. But the AGI talk seems to be mostly aimed at retail investors and philospher podcasters than institutional capital.
"With viable economics" is the point.
My "ludicrous statement" is a back-of-the-envelope test for whether an industry is nonsense. For comparison, consolidating all of the Pets.com competitors in the late 1990s would not have yielded a profitable company.
Do you argue in good faith?
There’s a difference between being too early vs being nonsense.
Not in the 1990s. The American e-commerce industry was structurally unprofitable prior to the dot-com crash, an event Amazon (and eBay) responded to by fundamentally changing their businesses. Amazon bet on fulfillment. eBay bet on payments. Both represented a vertical integration that illustrates the point–the original model didn't work.
> There’s a difference between being too early vs being nonsense
When answering the question "do the investments make sense," not really. You're losing your money either way.
The American AI industry appears to have "viable economics for profit" without AGI. That doesn't guarantee anyone will earn them. But it's not a meaningless conclusion. (Though I'd personally frame it as a hypothesis I'm leaning towards.)
OP did not include this requirement in their post because doing so would make the claim trivially true.
Other people just call it "theft".
I think this might be similar to how we changed to cars when we were using horses
Asking because, reading the tea leaves from the outside, until ChatGPT came along, MSFT (via Bill Gates) seemed to heavily favor symbolic AI approaches. I suspect this may be partly why they were falling so far behind Google in the AI race, which could leverage its data dominance with large neural networks.
So based on the current AI boom, MSFT may have been chasing a losing strategy with symbolic AI, but if they were all-in on NN, they were on the right track.
What part do you find hard to believe? That tech companies would send people to speak at a university's computer science functions?
Let me give you another one you'll think I'm making up: virtual reality was a thing back in the mid- to late-90s and people were confidently hyping it up back then.
even in pop-culture, see the movie Lawnmower Man.
At the very least, Ilya Sutskever genuinely believed it, even when they were just making a DOTA bot, and not for hype purposes.
I know he's been out of OpenAI for a while, but if his thinking trickled down into the company's culture, which given his role and how long he was there I would say seems likely, I don't think it's all hype.
Grand delusion, perhaps.
Definitely interesting to watch from the perspective of human psychology but there is no real content there and there never was.
The stuff around Mythos is almost identical to O1. Leaks to the media that AGI had probably been achieved. Anonymous sources from inside the company saying this is very important and talking about the LLM as if it was human. This has happened multiple times before.
so just understand there’s a lot of of us “insane” people out there and we’re making really insane progress toward the original 1955 AI goals.
We’re going to continue to work on this no matter what.
1) True believers 2) Hype 3) A way to wash blatant copyright infringement
True believers are scary and can be taken advantage of. I played DOTA from 2005 on and beating pros is not enough for AGI belief. I get that the learning is more indirect than a deterministic decision tree, but the scaling limitations and gaps in types of knowledge that are ingestible makes AGI a pipe dream for my lifetime.
Seems more like an incredibly embarrassing belief on his part than something I should be crediting.
He doesn't need to be right but it's not crazy at all to look at super human performance in DOTA and think that could lead to super human performance at general human tasks in the long run
Your position is a tautology given there is no (and likely will never be) collectively agreed upon definition of AGI. If that is true then nobody will ever achieve anything like AGI, because it’s as made up of a concept as unicorns and fairies.
Is your position that AGI is in the same ontological category as unicorns and Thor and Russell’s teapot?
Is there’s any question at this point that humans won’t be able to fully automate any desired action in the future?
We already have several billion useless NGI's walking around just trying to keep themselves alive.
Are we sure adding more GI's is gonna help?
...just please stop burning our warehouses and blocking our datacenters.
If you present GPT 5.5 to me 2 years ago, I will call it AGI.
neural networks are solving huge issues left and right. Googles NN based WEathermodel is so good, you can run it on consumer hardware. Alpha fold solved protein folding. LLMs they can talk to you in a 100 languages, grasp tasks concepts and co.
I mean lets talk about what this 'hype' was if we see a clear ceiling appearing and we are 'stuck' with progress but until then, I would keep my judgment for judgmentday.
Now our idea of what qualifies as AGI has shifted substantially. We keep looking at what we have and decide that that can't possibly be AGI, our definition of AGI must have been wrong
In some sense, this isn't really different than how society was headed anyways? The trend was already going on that more and more sections of the population were getting deemed irrational and you're just stupid/evil for disagreeing with the state.
But that reality was still probably at least a century out, without AI. With AI, you have people making that narrative right now. It makes me wonder if these people really even respect humanity at all.
Yes, you can prod slippery slope and go from "superintelligent beings exist" to effectively totalitarianism, but you'll find so many bad commitments there.
Science fiction from that era even had the concept of what models are... they'd call it an "oracle". I can think of at least 3 short stories (though remembering the authors just isn't happening for me at the moment). The concept was of a device that could provide correct answers to any question. But these devices had no agency, were dependent on framing the question correctly, and limited in other ways besides (I think in one story, the device might chew on a question for years before providing an answer... mirroring that time around 9am PST when Claude has to keep retrying to send your prompt).
We've always known what we meant by artificial intelligence, at least until a few years ago when we started pretending that we didn't. Perhaps the label was poorly chosen (all those decades ago) and could have a better label now (AGI isn't that better label, it's dumber still), but it's what we're stuck with. And we all know what we mean by it. We all almost certainly do not want that artificial intelligence because most of us are certain that it will spell the doom of our species.
https://www.noemamag.com/artificial-general-intelligence-is-...
I've been working with a startup, and I want to invest in it, and for the paperwork for that, all the nitty gritty details; instead of spending $20k in lawyers and a whole bunch more time going back and forth with them as well, the four of us, me, their CEO, my AI, and their AI; we all sat in a room together and hashed it out until both of us were equally satisfied with the contract. (There's some weird stuff so a templated SAFE agreement wasn't going to work.) I'm not saying you're wrong, just that lawyers, as a profession isn't going to be unchanged either.
There is a reason so many scams happen with technology. It is too easy to fool people.
Isn't this tautology? We've de facto defined AGI as a "sufficiently complex LLM."
However, I don't think it is even true. LLMs may not even be on the right track to achieving AGI and without starting from scratch down an alternate path it may never happen.
LLMs to me seem like a complicated database lookup. Storage and retrieval of information is just a single piece of intelligence. There must be more to intelligence than a statistical model of the probable next piece of data. Where is the self learning without intervention by a human. Where is the output that wasn't asked for?
At any rate. No amount of hype is going to get me to believe AGI is going to happen soon. I'll believe it when I see it.
And how will you know AGI when you saw it?
If this progress and focus and resources doesn't lead to AI despite us already seeing a system which was unimaginable 6 years ago, we will never see AGI.
And if you look at Boston Dynamics, Unitree and Generalist's progress on robotics, thats also CRAZY.
I don't know, maybe AGI is possible but there's more to intelligence than statistical next word prediction?
The 'predicting the next word' is the learning mechanism of the LLM which leads to a latent space which can encode higher level concepts.
Basically a LLM 'understands' that much as efficient as it has to be to be able to respond in a reasonable way.
A LLM doesn't predict german text or chinese language. It predicts the concept and than has a language layer outputting tokens.
And its not just LLMs which are progressing fast, voice synt and voice understanding jumped significantly, motion detection, skeletion movement, virtual world generation (see nvidias way of generating virutal worlds for their car training), protein folding etc.
Yes, and unless you are prepared to rebut the argument with evidence of the supernatural, that's all there is, period. That's all we are.
So tired of the thought-terminating "stochastic parrot" argument.
Can an LLM decide, without prompting or api calls, to text someone or go read about something or do anything at all except for waiting for the next prompt?
Do LLMs have any conceptual understanding of anything they output? Do they even have a mechanism for conceptual understanding?
LLMs are incredibly useful and I'm having a lot of fun working with them, but they are a long way from some kind of general intelligence, at least as far as I understand it.
They learned already a lot more than any of us will. Additinal to this, you have a prompt and you can teach it things in the prompt. Like if you give it examples how it should parse things, with examples in the prompt, it becomes better in doing it.
I would say yes they learn.
"Can an LLM decide" I would argue that you frame that wrong. If a LLM is the same thing as the pure language part of our brain, than the agent harness and the stuff around it, would be another part of our brain. I find it valid to use the LLM with triggers around it.
Nonetheless, we probably can also design an architecture which has a loop build in.
"Do LLMs have any conceptual understanding" Thats what a LLM has in their latent space. Basically to be able to predict the next token in such a compressed space, they 'invent' higher meaning in that space. You can ask a LLM about it actually.
Yeah for AGI we are not there yet and we do not know how it will look like.
After a bit of further refinement, we'll start to call that process "learning." Eventually the question of who owns the notes, who gets to update them, and how, will become a huge, huge deal.
It's not supernatural, I believe that an artificial intelligence is possible because I believe human intelligence is just a clever arrangement of matter performing computation, but I would never be presumptuous enough to claim to know exactly how that mechanism works.
My opinion is that human intelligence might be what's essentially a fancy next token predictor, or it might work in some completely different way, I don't know. Your claim is that human intelligence is a next token predictor. It seems like the burden on proof is on you.
Literally it is, at least in many of its forms.
You accepted CamperBob2’s text as input and then you generated text as output. Unless you are positing that this behavior cannot prove your own general intelligence, it seems plain that “next token generator” is sufficient for AGI. (Whether the current LLM architecture is sufficient is a slightly different question.)
And while I am typing, and while I am thinking before I type, I experience an array of non-textual sensory input, and my whole experience of self is to a significant extent non-lingual. Sometimes, I experience an inner monologue, sometimes I think thoughts which aren't expressed in language such as the structure of the data flow in a computer program, sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
When I solve difficult programming problems or other difficult problems, I build abstract structures in my mind which represents the relevant information and consider things like how data flows, which parts impact which other parts, what the constraints are, etc. without language coming in to play at all. This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
Therefore, my position is: it is plausible that next token prediction won't give rise to general intelligence, and I do not find your argument convincing.
COCONUT, PCCoT, PLaT and co are directly linked to 'thinking in latent space'. yann lecun is working on this too, we have JEPA now.
Also how do you describe or explain how an LLM is generating the next token when it should add a feature to an existing code base? In my opinion it has structures which allows it to create a temp model of that code.
For sure a LLM lack the emotional component but what we humans also do, which indicates to me, that we are a lot closer to LLMs that we want to be, if you have a weird body feeling (stress, hot flashes, anger, etc.) your 'text area/llm/speech area' also tries to make sense of it. Its not always very good in doing so. That emotional body feeling is not that aligned with it and it takes time to either understand or ignore these types of inputs to the text area/llm/speech part of our brain.
I'm open for looking back in 5 years and saying 'man that was a wild ride but no AGI' but at the current quality of LLMs and all the other architectures and type of models and money etc. being thrown at AGI, for now i don't see a ceiling at all. I only see crazy unseen progress.
I showed than counter examples.
"COCONUT, PCCoT, PLaT and co are directly linked to 'thinking in latent space'. yann lecun is working on this too, we have JEPA now."
Btw. just because you have to do something with the LLM to trigger the flow of information through the model, doesn't mean it can't think. It only means that we have to build an architecture around the model or build it into the models base architecture to enable more thinking.
We do not know how the brain architecture is setup for this. We could have sub agents or we can be a Mixture of Experts type of 'model'.
There is also work going on in combining multimodal inputs and diffusion models which look complelty different from a output pov etc.
If you look how a LLM does math, Anthropic showed in a blog article, that they found similiar structures for estimating numbers than how a brain does.
Another experiment from a person was to clone layers and just adding them beneth the original layer. This improved certain tasks. My assumption here is, that it lengthen and strengthen kind of a thinking structure.
But because using LLMs are still so good and still return relevant improvements, i think a whole field of thinking in this regard is still quite unexplored.
"In context" is the obvious answer... but if you view the chain of thought from a reasoning model, it may have little or nothing to do with arriving at the correct answer. It may even be complete nonsense. The model is working with tokens in context, but internally the transformer is maintaining some state with those tokens that seems to be independent of the superficial meanings of the tokens. That is profoundly weird, and to me, it makes it difficult to draw a line in the sand between what LLMs can do and what human brains can do.
Inability to introspect your own word selections does not mean it’s meaningfully different from what an LLM does. There is plenty of evidence that humans do a lot of things that are not driven by conscious choice and we rationalize it after the fact.
> I consider an entire idea and then decide what tokens to enter into the computer in order to communicate the idea to you.
And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
> sometimes I don't think and just experience feelings like a kiss or the sun on my skin or the euphoria of a piece of music which hits just right. These experiences shape who I am and how I think.
You cannot spin experience as intelligence. LLMs have the experience of reading the entire internet, something you cannot conceive of. Certainly your experiences shape who you are. This is a different axis from intelligence, though.
> This process seems completely detached from words. In contrast, for a language model, there is no thinking outside of producing words.
Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
> It seems self-evident to me that at least parts of the human experience fundamentally can not be reduced to next token prediction. Further, it seems plausible to me that some of these aspects may be necessary for what we consider general intelligence.
I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
This is correct and also completely irrelevant. I am describing what I experience, and describing how my experience seems very different to next token prediction. I therefore conclude that it's plausible that there is more involved than something which can be reduced to next token prediction.
> And how is that different? You are not so subtly implying that an LLM can’t consider an idea but you haven’t established this as fact. i.e. You are starting with the assumption that an LLM cannot possibly think and therefore cannot be intelligent, but this is just begging the question.
Language models can't think outside of producing tokens. There is nothing going on within an LLM when it's not producing tokens. The only thing it does is taking in tokens as input and producing a token probability distribution as output. It seems plausible that this is not enough for general intelligence.
> You cannot spin experience as intelligence.
Correct, but I can point out that the only generally intelligent beings we know of have these sorts of experiences. Given that we know next to nothing about how a human's general intelligence works, it seems plausible that experience might play a part.
> LLMs have the experience of reading the entire internet, something you cannot conceive of.
I don't know that LLMs have an experience. But correct, I cannot conceive of what it feels like to have read and remembered the entire Internet. I am also a general intelligence and an LLM is not, so there's that.
> Certainly your experiences shape who you are. This is a different axis from intelligence, though.
I don't know enough about what makes up general intelligence to make this claim. I don't think you do either.
> Both sides of this claim seem dubious. The second half in particular seems to be founded on nothing. Again, you are asserting with no support that there is no thinking going on.
I'm telling you how these technologies work. When a language model isn't performing inference, it is not doing anything. A language model is a function which takes a token stream as input and produces a token probability distribution as output. By definition, there is no thinking outside of producing words. The function isn't running.
> I don’t think anyone sane is claiming an LLM can have a human experience. But it is not clear that a human experience is necessary for intelligence.
I 100% agree. It is not clear whether a human experience is necessary for intelligence. It is plausible that something approximating a human-like experience is necessary for intelligence. It is also plausible that something approximating human-like experience is completely unnecessary and you can make an AGI without such experiences.
It's plausible that next token prediction is sufficient for AGI. It's also plausible that it isn't.
This is the fundamental issue. No one seems capable of defining general intelligence. Ten years ago most scientists would probably have agreed that The Turing Test was sufficient but the goalposts shifted when ChatGPT passed that.
If it’s not clear what AGI even means, it’s hard to say whether an LLM can achieve it, because it devolves into pointing out that an LLM is not a human.
The popularity of, and lack of consensus on, the Chinese room thought experiment kind of implies that this is wrong? I don't think many scientists (or, more relevantly, philosophers of mind) would, even 10 years ago, have said, "if a computer is able to fool a human into thinking it's a human, then the computer must possess a general intelligence".
Even Turing's perspective was, from what I understand, that we must avoid treating something that might be sentient as a machine. He proposed that if a computer is able to act convincingly human, we ought to treat it as if it is a human, not because it must be a conscious being but because it might be.
> the Chinese room thought experiment
This is an interesting thought experiment but I think the “computers don’t understand” interpretation relies on magical thinking.
The notion that “systemic” understanding is not real is purely begging the question. It also ignores that a human is also a system.
If what you are saying is true, then LLMs wouldn't be able to handle out-of-distribution math problems without resorting to tool use. Yet they can. When you ask a current-generation model to multiply some 8-digit numbers, and forbid it from using tools or writing a script, it will almost certainly give you the right answer. That includes local models that can't possibly cheat. LLMs are stochastic, but they are not parrots.
At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejorative sense you're applying to it. It can't be. The tokens in a transformer network are evidently not just words in a Markov chain but a substrate for reasoning. The model is generalizing processes it learned, somehow, in the course of merely being trained to predict the next token.
Mechanically, yes, next-token prediction is what it's doing, but that turns out to be a much more powerful mechanism than it appeared at first. My position is that our brains likely employ similar mechanism(s), albeit through very different means.
It is scarcely believable that this abstraction process is limited to keeping track of intermediate results in math problems. The implications should give the stochastic-parrot crowd some serious cognitive dissonance, but...
(Edit: it occurs to me that you are really arguing that the continuous versus discrete nature of human thinking is what's important here. If so, that sounds like a motte-and-bailey thing that doesn't move the needle on the argument that originally kicked off the subthread.)
(Edit 2, again due to rate-limiting: it does sound like you've fallen back to a continuous-versus-discrete argument, and that's not something I've personally thought much about or read much about. I stand by my point that the ability to do arithmetic without external tools is sufficient to dispense with the stochastic-parrot school of thought, and that's all I set out to argue here.)
Okay, what do you think language models are doing when they're not producing token probability distributions? What processes do you think are going on when the function which predicts a token isn't running?
> At the risk of sounding like an LLM myself, whatever process makes this possible is not simply next-token prediction in the pejoreative sense you're applying to it.
I don't know what pejorative sense you're implying here. I am, to the best of my ability, describing how the language model works. I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
What are you doing when you are not outputting tokens? You have a thought, evaluate it, refine it, repeat.
You’re not wrong that the basic building block is just “next token prediction”, but clearly the emergent behaviors exceed our intuition about what this process can achieve. We’re seeing novel proofs come out of these. Will this lead to AGI? That’s still TBD.
> I genuinely believe that a language model is, in essence, a function which takes in a sequence of tokens and produces a token probability distribution as an output. If this is incorrect, please, correct me.
The pejorative is that you imply this is a shallow and unthinking process. As I said earlier, you are literally a token generator on HN. You read someone’s comment, do some kind of processing, and output some tokens of your own.
I mean I do think sometimes even when not typing?
> Will this lead to AGI? That’s still TBD.
This is literally what I have been saying this whole time.
Since we agree, I will consider this conversation concluded.
I bet the guy has never contributed a novel thought that could be argued as moving something of magnitude forward. If that is the case he ought to stop writing as if he were capable of doing so - and therefore has no understanding of what true intelligence is.
This overestimates introspective access.
The brain is very good at producing a coherent story after the fact. Touch the hot stove and your hand moves before the conscious thought of "too hot" arrives. The hot message hits your spinal cord and you move before it reaches your brain. Your conscious mind fills in the rest afterwards.
I don't think that means that conscious thought is fake. But it does make me skeptical of the claim that we first possess a complete idea and only then does it serialize into words. A lot of the "idea" may be assembled during the act of expression, with consciousness narrating the process as if it had the whole thing in advance.
With writing, as in this comment, there's also a lot a backtracking and rewording that LLMs don't have the ability to do, so there's that.
Before you start typing, an fMRI machine can tell you which finger you'll lift first, before you know it yourself.
We are not special. Consciousness is literally a continuous hallucination that we make up to explain what we do and what we think, after the fact. A machine can be trained to behave identically, but it's not clear if that's the best way forward or not.
Edit due to rate limiting: to answer your question, the substrate your mind uses to drive this process can be considered an array of tokens that, themselves, can be considered 'words.'
It's hard to link sources -- what am I supposed to do, send you to Chomsky and other authorities who have predicted none of what's happening and who clearly understand even less?
This seems like a factual claim. Can you link a source?
(Also why respond in the form of an edit?)
it absolutely is a next word predictor
However, a much simpler explanation for what we see with LLMs is that instead the higher level encodings in latent space match only the patterns of our language(s), and no deeper encoding/understanding is present.
It's Plato's Cave - the shadows on the wall are all an LLM ever sees, and somehow it is expected to derive the real reality behind them.
At least Mythos model with its 10 Trillion parameter might indicate that the scaling law is valid. Its a little bit unfortunate that we still don't know that much more about that model.
Their progress is almost nought. Humanoids are stupid creations that are not good at anything in the real world. I'll give it to the machine dogs, at least they can reach corners we cannot.
I can also recommend looking at Generalist: https://www.youtube.com/@Generalist_AI
How can you say the advancements since Honda's asimo robot amount to "almost naought"?
There is a difference to be acknowledged: in the 70s/80s the whole world didn't suddenly start to shift to AI right?
So why do so many smart and/or rich people push this? Hype? Yeah sure but hype was here for crypto too.
I bet its an undelying understanding and the right time with the right components: Massive capital for playing this game long enough to see through the required initial investment, internet for fast data sharing, massive compute for the amount of data and compute you need, real live business relevant results (it already disrupts jobs) etc.
The necessary amount of Compute, interconnect (internet), money, researcher etc. wasn't available at that time.
and we did not invest the most amount of money and compute and brain power as we are doing right now. This is unseen.
"The new economy" also didn't have anything to do with the previous one. Turns out that it crashed just as well.
I do follow ML/AI/AGI though for a decade by now and read a lot about Neuronal networks, LLMs, etc. in a broad spectrum.
My prediction regarding Crypto/blockchain was true too.
We will see how it plays out. I'm open for both, but I think it would be naive to ignore whats going on and its way to soon to assume there is a AI winter coming soon.
We sitll want to see what Mythos can do and a distilled version of it.
is it? we're currently scaled on data input and LLMs in general, the only thing making them advance at all right now is adding processing power
Crypto was flawed from the beginning and lots of people didn't understood it properly. Not even that a blockchain can't secure a transaction from something outside of a blockchain.
LLMs don't have to be perfect, they just need to be as good as humans and cheaper or easier to manage.
$100+ billion in R&D and it's not comparable... hmm
And yet they don't do really good jobs with pretty much anything, save for software development, to which people still seem pretty split as far as it being a helpful thing. That's before we even factor in the cost.
I also believe that whatever code researchers and other non software engineers wrote before coding agents, were similiar shitty but took them a lot longer to write.
Like do you know how many researchers need to do some data analysis and hack around code because they never learned programming? So so many. If they know how to verify their data (which they needed to know before already), a LLM helps them already.
There is also plenty of other code were perfection doesn't matter. Non SaaS software exists.
For security experts, we just saw whats happening. The curl inventor mentioned it online that the newest AI reports for Security issues are real and the amount of security gaps found are real and a lot of work.
Image generation is very good and you can see it today already everywere. From cheap restaurants using it, to invitations, whatsapp messages, social media, advertising.
I have a work collegue, who is in it for 6 years and he studied, he is so underqualified if you give me his salary as tokens today, i wouldn't think for a second to replace him.
Than suddenly one model update moves it from 80% to 85% and now 30% of the market wants to use it.
Then it might be already too late to act like using it to your advantage, being a valuable expert or deciding things long term based on the new state of affairs.