undefined

points

[-]

Context is not the same as learning. It's easy to conflate because they're tightly coupled in our brains.

The underlying structure and tuning of the LLM are entirely unchanged by context. It merely affects the attention and activation of the network. The LLM will not be able to work with this hypothetical new language unless it is in context. This does not fit the computational meaning of learning.

Smart is not a well defined term. Nor is it's general idea formally understood. Use it freely, but you won't be saying anything meaningful unless you define your usage.

by xylenox2 hours ago|

prev|

[-]

Yep, people always forget that early LLMs were sold as "Zero Shot Learning".

by kmacdough1 hours ago|

parent|

[-]

Sold as learning, but that was a marketing term, not a technical one. From a technical perspective, the LLM is not learning. Only reacting based on its original training.

You might argue that the systems we've built around them are learning in a way, as they strategically condense and save artifacts from past interactions to pass into the LLMs context. But the LLM itself, which is the source of the intelligence, is not learning. It remains entirely unchanged throughout inference. This difference may seem trite, but it has significant impacts over the long term behavior.