I wouldn’t say they have an undefined truth value. Their source of truth is their training data. The problem is that human text is not tightly coupled to the capital T truth.
replyNor is the LLM output tightly coupled to the training data. They'll "eagerly"[1] fill in the blanks wherever it sounds good.
reply[1] here I don't mean to imply agency, just vigor.