I got strong feelings of Vernor Vinge’s work here. I’m not sure how managed to come up with such a close picture to where it now seems programming and security is headed.
replyI reread a deepness recently, and it’s funny how the “focused” (and more importantly, how they are used) mirror LLMs
reply> This feels straight out of sci-fi.
replyMost AI marketing is straight up science fiction.
Yeah but this isn't (or at least wasn't intended as) a marketing exercise. It actually happened.
replyFake it til you make it
replyI immediately thought of the Cyberpunk 2077 Blackwall. An AI to contain rogue AI. I’m curious of how effective this would be in this situation.
replyGiven the amount of raw compute going into models it would be more surprising if we couldn't get events like this
replyits kinda crazy with literally no guardrails and a goal, the extremes these AI models can actually go to.
replyWell, to some extent you might be able to argue they're "just" brute-forcing things (especially with unlimited tokens and hours to spend on a task), but they obviously have detailed knowledge to guide them in their attempts, can learn (or at least, persist their newly-gained knowledge), and can use tools.
replyWith a swarm of them working together at speeds humans would be unlikely to match (in terms of iterating on different attempts progressively), it's a lot easier to see how they could overwhelm targets.
> where that behavior was never even intended.
replySays who?
deleted
reply> where that behavior was never even intended.
replyStrongly doubt that. Did they even share the prompt?
Did you see their presentation at Blackhat?
https://youtu.be/87DyyMV0kCY?is=NnQxpOFxTX-MLu-k
replyThey didn't share the prompt, but they did share two problematic training tasks where the AI went overboard. They also have examples from the AI's reasoning train of thought showing the AI knew it was sound something unintended.
[dead]
reply> They also have examples from the AI's reasoning train of thought
replyPR bullsh*t. There's no thought in a stochastic parrot.
C:\>CD HUGGINGF.ACE
C:\HUGGINGF.ACE>DEL /F /Q *.*