I think it's a mistake to disentangle their abilities from understanding. Just swallow the pill that they have some form of understanding, even if it slightly differs from ours. I really don't see the problem.
replyI prefer to work the other way around. That is, accept that a lot of human speech (and text) is generated via similar mechanisms to the ones that drive LLMs, but note that there is another kind of behavior - reasoning - which seems to be distinct.
replyI think you need understanding to reason, but you don't need reasoning to understand. A child understands how to catch a ball without reasoning about forces, air resistance, gravity, etc.
replyI think LLMs understand without reasoning. They've built a large associative network of concepts (a kind of understanding), but we don't yet have a good handle on the process of reasoning using that network.
I don't think it is useful to say that a child "understands how to catch a ball", even though it is something many of us do quite often.
replyThe child knows how to catch the ball, without understanding. Later, the child learns both reason and physics, and can reason about ball catching in a different way.
I don't think that it is useful to say that LLMs understand anything they say, or that we say to them.