In my mind, hallucination is when some aspect of the model's response should be consistent with reality but is not, and the reality-inconsistent information is not directly attributable or deducible from (mis)information in the pre-training corpus.
While hallucination can be triggered by setting the temperature high, it can also be the result of many possible deficiencies in model pre- and post- training that result in the model outputting bad token probability distributions.
By "reality", do you mean the training corpus? Because otherwise, this seems like a strange standard. Models don't have access to "reality".
This is an explanation of why models "hallucinate" not a criticism for the provided definition of hallucination.
If the LLM is accurately reflecting the training corpus, it wouldn’t be considered a hallucination. The LLM is operating as designed.
Matters of access to the training corpus are a separate issue.
I want to say it was some fact about cheese or something that was in fact wrong. However you could also see the source gemini cited in the ad, and when you went to that source, it was some local farm 1998 style HTML homepage, and on that page they had the incorrect factoid about the cheese.
That would mean that there is never any hallucination.
The point of original comment was distinguishing between fact and fiction, which an LLM just cannot do. (It's an unsolved problem among humans, which spills into the training data)
No it wouldn’t. If the LLM produces an output that does not match the training data or claims things that are not in the training data due to pseudorandom statistical processes then that’s a hallucination. If it accurately represents the training data or context content, it’s not a hallucination.
Similarly, if you request that an LLM tells you something false and the information it provided is false, that’s not a hallucination.
> The point of original comment was distinguishing between fact and fiction,
In the context of LLMs, fact means something represented in the training set. Not factual in an absolute, philosophical sense.
If you put a lot of categorically false information into the training corpus and train an LLM on it, those pieces of information are “factual” in the context of the LLM output.
The key part of the parent comment:
> caused by the use of statistical process (the pseudo random number generator
also, statments with certainty about fictitious "honey pot prompts" are a problem, plausibly extrapolating from the data should be more governed by internal confidence.. luckily there are benchmarks now for that i believe
first compression: You create embeddings that need to differentiate N tokens, JL lemma gives us a bound that modern architectures are well above that. At face value, the embeddings could encode the tokens and provide deterministic discrepancy. But words aren't monolithic , they mean many things and get contextualized by other words. So despite being above jl bound, the model still forces a lossy compression.
next compression: each layer of the transformer blows up the input to KVQ, then compresses it back to the inter-layer dimension.
finally there is the output layer which at 0 temp is deterministic, but it is heavily path dependent on getting to that token. The space of possible paths is combinatorial, so any non-deterministic behavior elsewhere will inflate the likelihood of non-deterministic output, including things like roundoff. heck most models are quantized down to 4 even2 bits these days, which is wild!
If anything, I think all of their output should be called a hallucination.
Knowing is actually the easiest part to define and explain. Intelligence / understanding is much more difficult to define.
... To that end, I'd love to be able to revisit my classes from back then (computer science, philosophy (two classes from a double major), and a smattering of linguistics) with the world state of today's technologies.
If you pick up a dictionary and review the definition of "hallucination", you'll see something in the lines of "something that you see, hear, feel or smell that does not exist"
https://dictionary.cambridge.org/dictionary/english/hallucin...
Your own personal definition arguably reinforces the very definition of hallucination. Models don't get things right. Why? Because their output contrasts with content covered by their corpus, thus outputting things that don't exist or were referred in it and outright contrast with factual content.
> If anything, I think all of their output should be called a hallucination.
No. Only the ones that contrast with reality, namely factual information.
Hence the term hallucination.
https://patternproject.substack.com/p/from-the-mac-to-the-my...
He didn't hallucinate the Marriage of Figaro but he may well have been hallucinating.
Show HN: Gemini Pro 3 generates the HN front page 10 years from now
Generates does not convey any info on the nature of the process used to create the output. In this context, extrapolates or predicts or explores sound more suitable.
But nitpicking over these words is pointless and represents going off on a tangent. The use of the term "hallucination" reffers to the specific mechanism used to generate this type of output. Just like prompting a model to transcode a document and thus generating an output that doesn't match any established format.
You can easily say, Johnny had some wild hallucinations about a future where Elon Musk ruled the world. It just means it was some wild speculative thinking. I read this title in this sense of the world.
Not everything has to be nit-picked or overanalysed. This is an amusing article with an amusing title.
I'm partial though, loving Haskell myself (as a monad_lover) i'm happy it wasn't forgotten too :)
(“Generate”, while correct, sounds too technical, and “confabulate” reads a bit obscure.)
That is what the LLM are molded to do (of course). But this is also the insistence by informed people to unceasingly use fallacious vocabulary. Sure a bit of analogy can be didactic, but the current trend is rather to leverage on every occasion to spread the impression that LLM works with processes similar to human thoughts.
A good analogy also communicate the fact that it is a mere analogy. So carrying the metaphor is only going to accumulate more delusion than comprehension.
There are AI professors out there already!
Pronunciation: ex-tra-clau-dee-pos-TE-ri-o-ri-fa-bri-KA-tee-o
Meaning: "The act of fabricating something by pulling it from one’s posterior."
extra- = out of
claudi- (from claudere, close/shut) repurposed for “the closed place”
posterior- = the backside
fabricatio = fabrication, invention
Pronunciation: POH-ows-den-kungs-heh-RAUS-tsee-oongs-MAHKH-verk
Meaning: "A contrived creation pulled out of the butt by thinking it up."
Po = butt
Ausdenkungs- = thinking-up
Herausziehung = pulling-out
Machwerk = contrived creation
Pronunciation: POOKH-vo vang-DEE-moakh-CHU-ghakh (roll the gh, hit the q hard, and use that throat ch like clearing your bat’leth sinuses)
Meaning: "The perfected act of boldly claiming something pulled out from the butt."
puch = toilet (a real Klingon word)
-vo’ = from
vang = act, behave, assert (real root)
-DI’ = when (adds timing spice)
-moH = cause/make
-chu’ = perfectly / clearly / expertly
-ghach = turns a verb phrase into a noun (canonical nominalizer)