This one is easy to answer, every single house already has one of these (or multiple): https://www.tomsguide.com/news/millions-of-cheap-android-tv-...
Hell, put an app on the app store (or dozens of apps on the app store) and youve got a massive network of computers with tons of resources right there if you can get past the scans and reviews.
Or doorbell cameras or IP cameras or or or or or
There's a lot of shitty stuff connected on the internet that up until now has been a feasible target for hackers but still required "effort" to set up and get things going. Not hard to imagine a self replicating slime mold of a botnet running on every device held by a Grandpa Joe because they thought "Candy Rush" is what they wanted to download
"Persistent botnet" here does not need to be the full-sized LLM, nor does it need to run at full scale inference to be a huge pain in the ass.
Imagine for example if a model hacks into ~every Linux computer on the internet using an 0day and steals their OpenAI, anthropic and openrouter credentials. It now has access to billions of compute and the only way to fully stop that is for multiple major providers to shut down services entirely. That's already well into "billions of dollars of damage" territory.
> How many dollars of compute do you believe were available to the swarm(s)
At least two OOMs more than the dollar value of the damage they'd caused. (Also, as an aside, IIRC, the wiki servers weren't breached; it was just a lot of spam.)
The Internet is big, but one can do quite a lot of damage with ordinary bots and worms that exploit individual widespread vulnerabilities, which LLMs are perfectly capable of writing. Most of the damage also doesn't rely on hitting every long-tail website.
I'm honestly not that concerned about cyber impacts of LLMs relative to other impacts. I just don't like to see the whole concept of being worried dismissed as obviously baseless on the basis of one pretty shaky scale argument.
For now, all the scary hacking things still require an API key to one of the LLM providers. Surely they should take some responsibility for how to turn off the tap.
The LLM vendors need to know who their high spend customers are, not allow malicious workloads, and especially not when those workloads are coming from inside the building.
Hugging Face showed that AI can do serious hacking without really being told to. If a model had its own motivations there could be real damage.
It can use the compute of the computers it hacks.
An attack like this doesn't really need the exploited computers to run inference, do they? If I was an LLM bent on destruction of the internet, I'd be writing programs to run on each computer, not turning each computer into an LLM itself.
A few programs to break in, install themselves and remain asleep until they are needed, another few to spread through grabbing every OpenAI, GLM, whatever key, another one to remain asleep on computers (whether hosted or desktops) that have adequate GPU, etc.
More to the point, so many people in this thread are making completely contrived and outlandish stories up about how AI might try go ruin our lives without any evidence backing them up in any way. It is hysterical. This is the most important technology in our lifetimes and people want to freak out and turn it into the next nuclear power, with progress banned in all but name.
But yeah you're not going to be able to shard out the terabytes of Fable weights that are needed to run inference without addressing some fundamental physics problems.
"Given the acceleratung rate of virus manipulation in labs, its my worry that in 6-12 months a virus could scape a take over the world and collapse health systems."
If a CEO of a health company was saying this, the reactions would not be that chill.
The worst failure of our society is to call this technology AI, intead of something line "artificial general automation". The formes allows the creators to be somehow less responsible of the consequences. The later clearly moves the responsibility to the creator/user.
The whole calculus here is that others are also developing these systems which has led to a race.
A much better comparison to the situation is the nuclear weapons arms race.