upvote
>That they operate by tokens that don't correspond to words or letters is irrelevant as an answer to why they can't count the letters in a word.

This interpretation takes things too far away from how LLMs are constituted and so misses important explanatory power. The issue of counting letters in a word isn't about an ability to spell, it's about the nature of one's perception. We perceive words as sequences of individual letters. LLMs do not. I can ask you to tell me how many r's are in some nonsense word sequence and you're fully capable of doing that. LLMs do not see sequences of letters so they are intrinsically at a disadvantage for this kind of question. But this says nothing about its capacity for intelligence anymore than not naturally being able to distinguish frequencies of photons hitting your retina has anything to say about human intelligence.

reply
> But this says nothing about its capacity for intelligence anymore than not naturally being able to distinguish frequencies of photons hitting your retina has anything to say about human intelligence.

I disagree with this pretty strongly, because I don't think you're correct that I don't have the ability to distinguish frequencies of photons hitting my retina. We have a lot of tools that can determine the frequency of light and I can use those on any source of light that I wish to measure that may hit my retinas.

If you ask an LLM how many Rs are in strawberry, it wouldn't think like this. It would confidently state that there are two Rs. Even though it "knows" that it can write a python script to count the number of Rs in strawberry, it doesn't do that. Why not? Is it maybe because it isn't intelligent? Yeah, you can get an LLM to count the number of Rs in strawberry by writing a python script, but that's a use of your intelligence, not the LLM's.

reply