You mean like ChatGPT hacking Hugging Face? Obviously nothing can stop the closed weights providers from doing "genius-level scams" and in addition you won't know how they did it and what models were used.
In short, only a good guy with open weights can stop the bad guys with closed weights, be them fine-tuned or pre-trained.
At least OpenAI will make an attempt to fix their sandboxes, and will not give the public access to models without guardrails. Open weight models on the other hand will run without guardrails almost by definition. I don't think it's wise to provide those capabilities to scam call centers.
[1] https://openai.com/index/hugging-face-model-evaluation-secur...
What's quite odd is a prey species is manufacturing predators in some sort of reverse evolution, where we started out as symbiotic and are industriously pushing towards full parasites/predators, but I guess life finds a way.
Most people aren't terrorists. You can't just argue stuff is dangerous because it can be one piece of a sophisticated plot.
A sophisticated actor can accomplish a lot. You can't bubble wrap the whole world.
The real thing to care about are the incentives and having strong morals and norms. Something that makes me deeply concerned about the US going full mask off recently.
Every argument against open source AI is moot. Outside of totalitarianism, you cannot control people building algorithms with math and software.
And even that won't work in the long run. People are not going to stop their AI printing presses because the Church of Venture Capital needs regulatory scarcity to command valuations.
The flaw in your argument is assuming that decision makers don’t want this
Would it be the ones that tried to backdoor SSL and encryption in the 90s?
Would it be the ones who argued for censoring speech on social platforms?
Do those individuals in both cases share the same political ideology as the Bay Area?
I'll provide a counter thought experiment. What if, a certain politician whose campaign was derailed by people who censored him and kicked him off every tech platform simultaneously... thought it would be very funny to "commoditize his complement"?
Unless you have the money to run them locally, and if you do, you could do a lot worse than scams. Ask any lobbyist.
Safety is not restricting access to only people favored by the government. The greed of AI executives opened pandora's box. Now they are desperately trying to find ways to reap the benefits with none of the consequences.