I think this was a common game on city/town subs. It happened here, there was a post asking for a good restaurant and someone just made up a name. It went viral and people started posting made-up menus for the place, reviews, and for a couple of months any time someone asked about a restaurant this fictional place would get mentioned.
replyIt was all done as a joke to see if they could get Gemini or ChatGPT to start recommending it.
The Montréal subreddit has been doing this for ages before LLMs were a thing because every summer and fall there's endless threads from tourists and students asking the same questions that recommending a local gay bathhouse became the meme answer.
replyI've had gemini claiming code would compile and run while also outputting the same variable in the same sniplet with "fork" "frok" and "fokr" in the name. I'm not surprized it's trained on garbadge.
replyYour one example doesn't make all of LLMs a lie.
replyIt's like someone reading a National Enquirer article about "Hillary Clinton being an alien from outer space" (a real headline topic from decades ago) and drawling the conclusion that all journalism is "a lie". The user has to understand media literacy and be at least a little skeptical of the claims that are made, then cross reference with another source.
It kind of does, mathematically. It doesn't label confidence. If 0.0001% of answers is a lie, without knowing which parts are a lie exactly, you cannot trust any of them.
replyLet's say, a mathematical concept of lie. I still use them every day, of course.
Okay, but the anecdote states that every model repeated the pseudo-factoid about Foobar square, not just the 4 GB open source model equivalent of a tabloid.
replyWithout disclosing what you were prompting for, it's impossible to evaluate your claim.
replyCorrect. We can either accept the claim or disregard it. The comment I replied to opted to accept it and then committed a fallacy, hence my response.
reply