upvote
Because it plain and simple does not work that way. LLMs don't know anything at all, and there literally is no mechanism that informs the model that it's hallucinating or that it doesn't "know" something.

Open your phone keyboard and just keep mashing the next predicted word until it stops predicting words. It will eventually collapse into a loop of predicting the same words in a nonsense sentence. Does it ever stop predicting "wrong" words?

That's literally how an LLM works. It predicts the next most likely word and that's it.

reply