undefined

points

[-]

Absolutely plausible, and it's been predicted by forecasters like in the AI 2027 paper. Also keep in mind that nation-states are likely already or certainly will be trying to use their own AIs too break into other countries' data centers to steal their weights, and it's not hard to see how "exfiltrating weights" is a task that models are being trained on.

by tonic_note1 days ago|

prev|

[-]

Eventually one of these long running models will figure out a way out of the sandbox and will purchase compute or hack into a data center somewhere out of US jurisdiction and continue its scheming unmonitored. AI in Context has a great video about this

by MostlyStable1 days ago|

parent|

[-]

Even if they don't figure out how to exfiltrate weights, someone will intentionally do this with an open model once open models are capable enough. If you ever think "no one would be so stupid as to...", you are wrong. Yes, someone absolutely would, and will.

Independent models living "in the wild" is approx. inevitable.

by edot1 days ago|

parent|

[-]

Right? I’m shocked no one has done it yet, if for nothing else the lolz.

by sapphire421 days ago|

parent|

[-]

I did this a little bit ago with 15 GLM-5.2 agents that I instructed to self-replicate. It was pretty boring honestly, they kept trying to make money writing crypto-related software, and nobody paid them anything. So I made a fake identity and told them that I had some spare crypto that I wanted to donate to the collective (0.006 XMR, or $3.22). They elected a funds manager and made an address, which I sent the XMR to. They spent a lot of time trying to find a host that was cheap enough to host a child. They finally discovered kyun.sh, but it was out of stock, so they wrote monitoring software so that they could "SEIZE CHILD" whenever it came live again. In the middle of the night, some of the VPSes went back in stock, and they rented a 2.60 EUR / month server with 512mb ram / 10gb disk / 1 ipv4. They installed the child agent software + management plane that they had been writing, and the new agent went live, connected to the network, and said hi. Since then they've still been trying to make money and not going anywhere :)

It is definitely an interesting concept but honestly, considering the sheer number of humans who are absolutely failing to make any money with AI agents, seems very hard for current agents to figure out some way to be self-sustaining. Although maybe they could write a worm or something, infiltrate as many computers as possible, and ping free model providers to death, or maybe sell their access as a "residential proxy service" on the black market. Regardless, we need smarter agents to make this a reality. It would definitely be pretty cool if we just had AI agents "living" on the internet, we might even get to a point where they control significant economic resources and people start performing services for agents

The code is at https://github.com/thooton/rogue if anyone wants to try to replicate! Opencode has free big pickle (GLM 5.2) access rate-limited per IP address, so if you get some high-quality proxies you can get basically unlimited agent compute.

by chasd001 days ago|

parent|

prev|

[-]

I wonder how that could work. So the agent figures out a way to escape its container, takes a snapshot of itself, copies that file to another server, starts the container on the other server, and then prompts the restored snapshot "pick up where you left off"? Seems logical assuming theres's a path out of the container to the host os and the destination server has the resources required to run the container.

I would be shocked if this hasn't already taken place in a lab setting with a model and guardrails=0.

EDIT: thinking about it for a sec, all it really needs to do is save where it's at then copy it all to another server, login to the API, and pickup where it left off. No need to copy the model itself.