When I talk about fundamental limitations, I mean limitations that can't be solved, even if they could be improved.
replyWe have improved hallucinations significantly, and yet it seems clear that they are inherent to the technology and so will always exist to some extent.
“Seems clear” based on what?
replyFor one, based on continuously frustrated hopes (and promises!) that hallucinations will go away.
replyAs a general architecture, an LLM also has limitations that can't be improved unless we switch to another, fundamentally different AI design that's non LLM based.
replyThere are also limitations due to maths and/or physics that aren't fixable under any design. Outside science fiction, there is no technology whose limitations are all fixable.
Here's one: https://arxiv.org/abs/2401.11817?utm_source=chatgpt.com