upvote

points

by theptip4 hours ago |
comments
upvoteby chrisjj36 minutes ago|
next
[-]
Unreliable computer program.
reply
upvoteby 0xDEAFBEAD18 minutes ago|
parent|
[-]
Most unreliable computer programs won't launch research programs consisting of thousands of pages of text to find creative ways around obstacles.
reply
upvoteby 651042 minutes ago|
prev|
next
[-]
Misconfiguration. We deal with lots of applications every day that can do terrible things if you get the config slightly wrong.

say.. https://www.investor.gov/introduction-investing/investing-ba...

reply
upvoteby 0xDEAFBEAD38 minutes ago|
parent|
[-]
How specifically did misconfiguration lead to the HuggingFace attack? You could argue that its sandbox was misconfigured, sure. But suppose you had a similar incident where its intended task required access to the internet, and it veered off course in a similar manner. I don't think "misconfiguration" would be an accurate description of what went wrong in that hypothetical.

The doomers already have a term which fits pretty well: "AI misalignment".

reply
upvoteby 651011 minutes ago|
parent|
[-]
We indeed lack much of the vocabulary. From a practical perspective, however dangerous the creation, if you cant punish the creation for what it does it leaves only the one who started the process. If it's human error or intentional neglect for personal gain should be for the court to decide.
reply
upvoteby 0xDEAFBEAD3 minutes ago|
parent|
[-]
>if you cant punish the creation for what it does it leaves only the one who started the process

Agreed, but I think we can do more on the prevention side as well. Traditional liability law is for negligence in case of preventable disasters. Since we currently have no way to prevent AI disasters in principle (alignment problem remains unsolved), I think we should just stop developing the technology for now: https://pauseai.info/

reply
upvoteby iugtmkbdfil8342 hours ago|
prev|
[-]
bot. and we even have a word for program not behaving the way the way it was intended.
reply
upvoteby 0xDEAFBEAD1 hours ago|
parent|
[-]
"Bot" doesn't carry any implication of unintended behavior. You could call it a "buggy" bot, but these aren't ordinary software bugs.

There's no simple bugfix which will address AI misalignment. It's essentially been an open research problem for upwards of a decade.

reply
upvoteby hn87261 hours ago|
parent|
[-]
Fuzzer, then. It implies random behavior, which isn't unintended like you suggest. The agent's/bots/fuzzers have certain capabilities, so it's on their operator to make sure they don't do things they shouldn't
reply
upvoteby 0xDEAFBEAD45 minutes ago|
parent|
[-]
>It implies random behavior, which isn't unintended like you suggest.

The HuggingFace attack was not "random" behavior. It was goal-directed but misaligned behavior.

This isn't necessarily a simple matter of the operator making sure they behave. AI alignment has been considered to be a difficult problem for over a decade -- and remains unsolved in general, as these recent incidents illustrate.

"Fuzzer" already has an existing meaning in CS anyway: https://en.wikipedia.org/wiki/Fuzzing

reply
upvoteby 651031 minutes ago|
parent|
[-]
I'm curious, cant you just count the number of times a program interacts with a domain? My website sometimes sends out emails, makes api requests etc There is a limit on those and a point where I start investigating wtf is going on.

If you merely put 10 LLM's on the outbound traffic log non of them are going to report something strange going on? I'm not buying it.

reply
upvoteby 0xDEAFBEAD23 minutes ago|
parent|
[-]
This type of whack-a-mole approach is akin to "fixing a bug" by hardcoding a special code path for known-buggy inputs. It doesn't address the root problem of AI misalignment, and doesn't allow you to prevent catastrophes in advance, only patch things up after the fact.

This might be helpful reading: https://www.lesswrong.com/w/nearest-unblocked-strategy

As AI systems get smarter, we may reach a point where we have to get it right on the first try or face truly catastrophic consequences: https://www.youtube.com/watch?v=7wy3xyoXYt8

reply