We can rank them based on how much they know and people will gravitate towards those that do know more.
It's a market after all.
Haven't heard about that law, but seems unlikely we can come up with ("discover") any sort of law that uses a concept ("truth") humans can't even agree what it means, and that's not for a lack of trying, we've been trying to figure it out for millenniums already with no end in sight.
Might be why we're already rarely seeing models output an "I don't know".
"Confidently incorrect" has negative value. At best, a human realizes the answer is wrong and At worst, the incorrect information makes is not identified and can cause untold damage. By having the potential to be so severely wrong, it lessens the value of correct answers because there is a lower confidence value on their output.
If someone sold you a "Solved all your problems" machine, and it suddenly doesn't solve all your problems, then probably no, you shouldn't pay.
But the way I'm being sold LLMs, is basically "A text generator that gives your plausible-sounding human text that sometimes hallucinates and gets things wrong, based on your input", then regardless of what the outcome is, I still made use of the "Input > Output" part, which is what I bought into, so I should still pay for that.
Now of course bunch of people will say they been sold the former, but the companies themselves seem to be selling the latter. That's my perspective from a person who doesn't follow "influencers" and what not though, which seem to be selling the public on the former rather than the latter.
My ask:
> In a couple sentences, explain to me the product I'm being sold with ChatGPT. What does it do for me?
The Reply from ChatGPT:
> ChatGPT is a conversational AI that helps you think, create, learn, analyze, and get things done faster. You can use it to answer questions, draft and edit writing, summarize information, brainstorm ideas, learn new topics, write code, plan projects, and increasingly act as an assistant that can search for information, work with documents, generate images, and help complete tasks.
> In simple terms: you're buying access to an AI that turns natural language into useful work—saving time, expanding your capabilities, and giving you an always-available collaborator for both everyday tasks and specialized knowledge work.
This sounds much more like the former, a "solve all your problems" machine.... not a plausible-sounding text generation machine.
Only two weeks ago Sam Altman said their new data center "could" be where cancer gets cured[0]. It is only the people who deeply understand AI who see it as a text generator of plausible-sounding text. That isn't what the marketing department, the CEO, or the product itself seem to be saying. I'm using OpenAI as the example here, but the others don't seem much different.
> Can I trust the output you give me?
And I assume it explains what to trust VS not.
I think in the bottom you should also see something like "Any text can contain mistakes" or similar too, which I know is a far cry from what some people push in the press in regards to capabilities, but I still don't see the platforms themselves as lying about this, while I do see a bunch of people constantly over-hyping the possibilities.
I'm not sure why "can I trust the output you give me?" would be a logical followup to the first response it gave me, seeing as it's response didn't say anything about hallucinations or mistakes. It said it could do "useful work" with all kinds of examples, including "specialized knowledge work".
The note under the text field, in gray as to not draw the user's attention, feels more like a CYA line from the lawyers, rather than an instruction they really want users to take to heart. That line also doesn't appear on the main home page. I only shows up after the first prompt is submitted and focus shifts to the conversation. I don't think a CYA line in gray fine print is enough to make users understand it's a plausible-sounding text generation machine instead of an answer machine. Even if I ask that point blank it gives a wordy... yes, but not really, it's being debated by philosophers... response.
> If you can dream it, Claude can help you do it. Claude can process large amounts of information, brainstorm ideas, generate text and code, help you understand subjects, coach you through difficult situations, simplify your busywork so you can focus on what matters most, and so much more.
What marketing copy have you read for LLMs that is like you mentioned?
> But the way I'm being sold LLMs, is basically "A text generator that gives your plausible-sounding human text that sometimes hallucinates and gets things wrong, based on your input"