undefined

points

[-]

I'm not offended by the line of questioning! But I don't really follow it. I don't and IIUC Bender doesn't use "understanding" to refer to any kind of magical property. Understanding is the capability of using words as consistent handles to things in the exterior world which the language is describing. And this is something LLMs can clearly do. I just went to ChatGPT and asked this question, which is almost surely not in its training data:

> What would happen if I walked to the top of a skyscraper with a soda can full of Maraschino cherries and let them go?

And its answer (https://chatgpt.com/s/t_6a4bd9ffa5708191901bb6d43c89f43b) clearly demonstrates understanding. It knew that this is a dangerous thing I should not do in real life, and that my question is ambiguous about whether I intend to drop the can, and that this might be intended as a physics problem rather than a real life scenario.

by 271832 hours ago|

parent|

[-]

> And this is something LLMs can clearly do.

...

> It knew that this is a dangerous thing I should not do in real life

From the ChatGPT response you linked, all I see for sure is some matches on the following patterns:

  drop $thing from skyscraper --> bad behavior
  drop $thing --> physics
  can of $stuff --> contents in/out of can

Then there are some sentences of likely characters following those patterns. You don't need anything more than a basic cartoon-level understanding of how an LLM works to explain this output. I see no evidence of reasoning or understanding here, or any theory of "real life".

It also does an incredibly poor job of answering your question. It makes no attempt to explain what might actually happen. If it has been trained on the entire corpus of medical science, and it is indeed intelligent, then surely it can reference ballistics studies and give you a very detailed and thorough theory of what--exactly--injuries you might expect from a 12oz can being dropped from the height of a skyscraper. Calculating the terminal velocity and therefore the momentum of the can is trivial. Characterizing the physics of the impact on various parts of a human body is trivial. If it actually understood your request why didn't it just answer the question?

It's a rhetorical question. LLMs do not "understand". It is completely outside their capability. "Understanding" is something we impose upon their output (to loosely quote TFA). [edit] I think the most powerful evidence for a lack of any understanding whatsoever is all the stuff about the cherries being in or out of the can. Yes, cans contain things. That is not a profound observation, nor is it at all relevant to the question. If you drop an empty can off a skyscraper nothing meaningful will happen. And, no, probably dumping all the cherries out won't hurt anyone or cause a slipping hazard... It's also not particularly relevant to point out that dropping things off skyscrapers is bad behavior. But that's more forgivable from a CYA standpoint.

I believe you are projecting something that is not there onto a completely mindless stochastic process.

[another edit] I found this helpful, you may also: https://www.0xkato.xyz/how-llms-actually-work/