Remember when people said Artifical Intelligence woun't be dangerous, because nobody will be stupid enough to give it free access to the internet...
Can't tell if you're saying this tongue-in-cheek or you're a bit out of the loop on what people are doing with LLMs.
And a quick correction:
> unless someone, somewhere is irresponsible enough to connect an LLM to something that actually matters.
The need to acquire expertise and/or a meaningful following has always been a significant impediment to malicious or moronic actors. But less so every day.
It is quite hard (but not impossible) to get an the frontier AI to tell you how to build a nuke or launder money now, where jailbreaks used to be trivial “ignore all previous instructions”.
It seems like a worthwhile effort.
In my opinion, these companies should put their effort elsewhere. Obviously if all someone is doing on their platform is looking up how to build a nuke, where to buy uranium, the best city to explode it in, etc. please report them to the authorities. If someone is clearly just using LLMs to write hate speech they go post on the internet, ban them. And so on.
This cat & mouse game trying to have LLMs police inquiries is ridiculous to me.
Yes, and: the LLM is a "brain in a jar". It doesn't have any ability to verify ground truths outside itself, other than maybe calling out over the internet. Therefore it is easy for humans to lie to. You could call this an "Ender's game" attack, after the book in which a hyperintelligent kid is playing "war games" that end up being the real war.
> The idea that an LLM can discern intent on any given prompt is farcical.
Not really though. For most people in most situations it's just not going to give you that info. Software security is a niche where its a bit strange in that there is 100X the amount of white hat users than bad actors and there's open source etc.
And ya, it's pretty easy to hide your intent once you have access.
KYC for example does stop most money laundering and financial crime. The most resourced actors like governments/ cartels often find ways around and it is a game of cat and mouse. Normal citizens don't really stand a chance to get around most of them.
Like it feels like your logic is that we shouldn't do background checks for employment because North Korean spy agencies get past them sometimes?
Clearly, there's no such thing as a perfect exclusion rule at any of these scales, but the false-negative to false-positive ratio seems like it will be way higher if Anthropic starts trying to verify IDs.
Or, much more likely, the same pattern of tokens happen to exist in a completely different discussion, either as a direct metaphor, or as a reality of linguistics. Hell, "laundering" itself is a metaphorical word.
The absurd notion is that any speech should be policed in the first place. If there really is such a thing as dangerous information, then it must be removed from the training data. Any other strategy simply launders the risk.
No security is ever perfect, but we can likely protect LLMs with WAFs that increase security to an acceptable level. Like nation-state required resources to break.