The LLM has to compress everyy question/prompt into its system. It does so by creating rules and ways of processing data (this can lead to AGI, world models or an architecture of sub architectures like an LLM + something else). So if it should respond in a way that only reasoning people can achieve, it might be able to learn a representation of what we call reasoning.
It read enough text in itself to even know about the concept of reasoning and how you would do that.
Even if this is only stochastic, it shouldn't be so devalued as your comment comes across.
Who says that we are doing anything more magic?
If that works, I think it's fair to say that LLM's are inanimate processes that can generate real reasoning. You can tell when you read it and it makes sense.
There are likely some kinds of reasoning that can't be written down, as well as other forms of understanding, but they also don't replicate nearly as easily.
1. phenomenal reasoning, requiring consciousness and subjective experience
2. functional reasoning, transforming premises into conclusions using logic
I think you are attacking this using definition 1, whereas the article is obviously aiming at a different type of reasoning, and trying to formalize what is actually going on. It seems to be a genuine effort.
I think it is incumbent upon anyone arguing that something does not posses any given property to provide a non-circular definition of what it is that they are declaring an absence of.
All of the descriptions of experiential reasoning are usually defined in terms of rephrasing of the claim "true understanding", "conscious", "aware", "knowing" all hinge on a synonymous aspect of the words that try and shift the responsibly of explanation to the next term used in a cyclic manner.
For the weaker sense of reasoning, there simply isn't any argument that it is not happening. A calculator can perform the weaker sense. The analysis of this aspect of LLMs is purely a question of how, not what.
This needs to be routine to be given asevidence…
…Unless you know exactly how the llm was trained and then how it was applied
It is a claim that swimming is a word that defines a context. It is an explicit statement that the question of whether a submarine can swim has nothing to do with the capability of the submarine.
If you are asking which pigeon hole we are putting something into, the answer is "The one we put it into". This is what make the question uninteresting.
If you are asking what is it about this pigeon hole that people value and does that align with the criteria that people use to decide categorisation. That very much is an interesting and complicated question.
I wonder if it is the same for programming or not, but I vibe coded an android app just to see if I can and it just works. It required a lot of "build the code and correct the errors" pushing though. For example requested code in kotlin but received something else.