> The only thing a ban would do is protect the American market from further downward price pressure on inference, protecting VC investors in the short term. But thats also an admittance that the American labs can’t compete on merit anymore
That's the argument. To be precise the publicly stated argument is that they're attacking American providers by distilling. The real aim is to eliminate competition because otherwise Anthropic and OpenAI are non viable and the US views them as crucial for winning the 'AI race' which they see as putting whoever wins it on top in terms of warfare/economic power etc.
Distilling or not, they are clearly close enough to the frontier, that the supposed „free market“ country needs market controls. That may work for US markets. But not the rest of the world.
Idea: petrodollar policy becomes „tokendollar“, enforced by US military dominance. If you force me to pay altman at gunpoint, then maybe ill stop using kimi
That's it. Simple as that. When you grasp for straws in panic mode, you don't exactly spend time strategizing and weighing the pros and cons of each straw carefully.
As far as terms, I don't like really like either side we have on the menu right now.
Good scare quotes. The US hasn't been free market in the Adam Smith sense in ages, if ever.
and because Chinese models are open-weights, the distillations effects of them could be easier done as well ;)
In my opinion, its a win-win plus even within worst case scenario*, I already believe that the current open weights models are in general speaking good enough perhaps for my and other use cases as well and I feel like we will probably most likely get more open-weights model for a long time in general as well perhaps.
> Someone please enlighten me what the actual argument here is, cause I can’t see it.
But you did see it.
The US has invested trillions in AI that the companies involved are never going to make back. Even without competition from open models, but definitely not with it. And if those trillions turn out to be worthless, that's going to have a massive impact on the market and cause a lot of bankruptcies.
Banning those open weight models isn't going to fix everything, but it would the impossible obstacle slightly smaller.
This is exactly it.
Try to buy a BYD in the United States. You can't (without a complicated process) because they're so much better cars than our domestic brands that our domestic brands couldn't compete and lobbied to keep them out.
> should by itself also limit the viability of the idea that all those VC billions will ever make a return?
It just has to last until the next quarter / fundraising round.
I am befuddled when the same policies are applied to products with marginally zero manufacturing cost, no real physical size, and such. Regulating ideas is hard folks.
Easy: you're a traitor, if you use foreign AI for the fraction of the cost. It's unamerican not to fill the American billionaires pockets. Free markets exist, as long as you buy America first!
... Yes I would.
It's free real estate.
Easily misremembered and meme-able.
This is precisely what the people worried about AI risk and job displacement want, so they should be strongly in favor of open weights then, right?
I just don’t buy it. There’s still gonna be demand for stronger models. Sure growth will be slower, but this might even drive a push for more cost efficient training / inference and/or new architectures if money is harder to come by.
It also might end up pushing us towards LLM ASICs assuming the model progress slows significantly.
potential issue is those will be Chinese models if they win, which could be national security matter.
It won't even do this. Streisand effect will probably draw even more attention to the open weight models
The entire marketing narrative in AI already operates this way --- "GPT 2.0 is too dangerous to release, oh noooo!" etc
https://en.wikipedia.org/wiki/Phil_Zimmermann
After a report from RSA Security, who were in a licensing dispute with regard to the use of the RSA algorithm in PGP, the United States Customs Service started a criminal investigation of Zimmermann, for allegedly violating the Arms Export Control Act.[5] The United States Government had long regarded cryptographic software as a munition, and thus subject to arms trafficking export controls. At that time, PGP was considered to be impermissible ("high-strength") for export from the United States. The maximum strength allowed for legal export has since been raised and now allows PGP to be exported. The investigation lasted three years, but was finally dropped without filing charges after MIT Press published the source code of PGP.[6]
AI labs and investors are scared, and pushing administration to ban them, because they can't compete with Chinese models soon, similar to how they banned Huawei, and Chinese cars.
When there is cheaper alternative, companies might go with self hosting option, which reduces the enterprise moat of AI labs
And that will go as well as it went for both
It wouldn't surprise me if that kind of problem is partially why weights are open. Because you can't meaningfully stomp them out completely.
That’s what I don’t get tho, I think in today’s world they totally could?
„We need to protect US investments as our entire economy depends on this market sector“ seems like something that would be soothing to me as an investor?
It’s not like the big money givers can’t see that china is close, it’s been the news multiple times already. But taking active policy steps to protect against that should boost market confidence, no?
Instead we get alibi arguments to accomplish the same thing in the end?
'We must protect VC profits at the cost of the rest of the economy and our entire 'free market' system' probably doesn't sound as good.
This administration has done a lot for that minority already so I don’t think anyone would be too surprised? In this case they can even claim to protect the greater economy, a crash of which does impact the majority.
„US good, China bad“ has been a long term talking point by Trump too and this would neatly fit into that to be consumed by his base
It's hilarious because at this point it's clear they can say that. We've just had a coordinated talking points push that voter picture IDs should be required because "Olive Garden".
I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".
It's not even going to be a good honeypot - since said actor doesn't really provide inference, people will have to pay for and run the infrastructure of these models, which bad guys cannot even subsidize, unlike cloud based provides.
is this even doable? it seems plausible the model can be order-66'ed without it being so obvious
Beyond certain obvious 'third-rails' even experts have reasonable disagreements on what 'aligned' means.
> everyone integrates it into their business
Few large corporations would sole-source integrate a closed model developed in a country designated as a foreign adversary (which could cut off access at any time) or which their own country might restrict.
> now the foreign actor has control
Your concern is one reason why the Chinese are making most of their models open weight.
> it's not as trivial as "regulatory capture".
You're right, it's not just regulatory capture but the vested interests trying to achieve regulatory capture are certainly leveraging concerns like yours to deceptively achieve that capture. In the case of an open weight, domestically-hosted model, the threat vector you're concerned about isn't a threat in the same way as a Chinese-made, closed source data center router or 5G phone switch.
1. https://www.wsj.com/tech/ai/anthropic-in-talks-to-invest-200...
Seems a bit far fetched, given that I’ve never seen any PoC/research on this topic. Got any pointers?
To be fair the fix to this seems to be that American labs would need to adjust to the new „fair market price“ then, right? Subsidy-backed competition is still legitimate competition in capitalism (e.g. Uber).
And you could still ban specific, known malicious models/labs instead of blanket open-source bans. What about Mistral models, for example.
An LLM is a collection of inert data, it is not an automaton.
You obtain an automaton only using an inference program, like llama.cpp or anyone of the many others, and then using some harness around the inference program.
You can put in the harness all kinds of guards, to detect any possible harmful action and prevent it.
In something like a CPU from Intel or AMD it is easy to put some undetectable backdoor that would allow the remote control of the computer, unless you disconnect any antennas and you filter all wired network traffic through an external firewall that would allow only whitelisted connections.
On the other hand an LLM is a much less plausible attack vector, as long as you host the inference yourself. Only an LLM that runs externally, e.g. at OpenAI or Anthropic, can be really dangerous.
> And you could still ban specific, known malicious models/labs instead of blanket open-source bans
With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)
I agree, I even think we’re already there if yesterdays OpenAI/HuggingFace story is legit.
We need to start building these agent systems in a way so that it won’t matter if a model is compromised. Malicious models is one way, but prompt injection is also still an unsolved problem.
And if you solve that, then a malicious model is also no longer a threat. Data exfiltration maybe, but IMO US labs are probably also doing that for training, even if saying otherwise (of course I have no evidence of that, but the temptation must be insane)
By supposing, you can try to justify taking just about anything away.
Let's not forget that as of now most models are like game cartridges i.e. they need a 'host' to run i.e. llama.cpp - they're not executable code by themselves.
I could see an argument for a security review prior to integrating a model into a highly sensitive industry/agency, but that is standard for any software.
It's certainly not an argument for banning open models because US AI corps don't want to compete with them.
I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.
Merit was never competitive. Mass consumption habits select hard for cost, and this is an industry driven entirely by scaling factors. If you can scale for cheaper, you win. Ivy League-decorated researchers aren't doing RnD for free. The frontier labs can only go so cheap before they're so far in the red that VC gets cold feet. Leaving the expensive frontier research up to private enterprise wasn't a long-term strategy.
> I don’t get it. Am I missing something? The only thing a ban would do is protect the American market from further downward price pressure on inference
What's not to get? Do you understand how much money is tied up in the industry? Do you know what happens if that goes up in flames? In the middle of a massive asymmetric economic depression? Why on earth do you think they would just sit there and watch it happen? Because of political virtue signaling about free markets? You know, I have a bridge to sell you if you're interested.
I think in capitalism thats called a market correction?
> Do you understand how much money is tied up in the industry?
If the free market „virtue signaling“ no longer matters, then they could’ve also put a stop-gap on investment volume to protect the economy against over-reliance?
Or how about go full plan-economy and just dictate price per token until investors get their money back?
I’m joking around but of course I understand whats tied up in this. I just don’t see how banning open source models is helping any of that in the long run.
Which instantly collapses the load bearing bull market. They can't do that for the same reason they can't do anything about the insanity of the real estate market. The fallout would be existential. The economy isn't rational, remember. It's highly reactive and, no matter how strong your state is, always out of your control at the end of the day.
America is years away from being able to rotate meaningful volumes of capital into heavy industry. Until then, AI has to outpace the cooling of the tech sector, while also contributing to it. The entire political strategy in play is a frantic spinning plate game. There is no long run if it fails, so that is their strategy as far as we can see. Or maybe they're just maliciously incompetent. Given the American administration opaquely decided starting a forever war in the Red Sea was a solid choice at this point in time, possibly to try and fluff up an auxilliary bull market with the MIC, we really can't rule that one out.
Just like unlicensed dupes of designer goods can be intercepted at the border.
These people paid a lot of money to Trump and now they want to cash in on that bought influence.
I admire China for its cunning, having put the US on this spot in their long-going economic war. If unchecked, it’s in China’s interest to release to the world for free models that can compete if not surpass the frontier ones.
It’s more an admittance that the capitalist model can lose to a more socialist approach, unless you “fight fire with fire” and use the government to distort the market for you.
Tl;dr, it's purely an economic argument. Every major argument in the world is about money and thus power, don't bother analysing it technically.
But the argument isn’t about model providers tho right (providing inference)? Using Chinese providers is already a privacy problem if user data is involved, so I don’t know if anything would change here.
What you’re suggesting is that the policy would stop big co‘s from running their own inference completely, no? Or only the foreign open source models?
> don't bother analysing it technically
I feel you have to, because where’s the line? Is running GPT-OSS myself fine (if I were a big company)? Mistral models? Or are none of them ok? What about if I am getting inference from a small startup, which is running OSS models for me? Or is only OAI/Anthropic ok?
Among the chiefs of the Fortune companies there is no complete unity, as some think that they should receive some of the AI profits, instead of OpenAI and Anthropic.
Recently, both the CEO of Palantir and then the CEO of Microsoft have criticized quite harshly the business model of OpenAI and Anthropic, implying that it is bad both for society and for their corporate customers.
None of those two would come first in mind for altruistic behavior, so their criticisms must be based on business reasons.
For the CEO of Palantir, it is clear that the criticism was a sales pitch for a new Palantir product (Sovereign AI), which is a turn-key system including all hardware and software needed by a company to self-host AI model inference and also training or fine tuning.
It is less clear which is the motivation of Microsoft. If they do not like the business model of OpenAI and Anthropic, then this means that they also want to compete with them, but that cannot be the same as what Palantir proposes, i.e. self-hosting on premises. Perhaps Microsoft also wants to propose some kind of self-hosting, but instead of on premises, doing it on some kind of Azure instances.
So the Chinese are not the only "enemies" for OpenAI and Anthropic.
A lot of elites are now unemployed with nothing better to do thanks to AI. Historically speaking, populist uprising are easily squashed except when you have a group of counter-elites supporting the movement. We're finally starting to see the results of that with the Deflock movement cutting down cameras.
If these AI companies tank the US economy, people will literally be cutting off Sam and Dario's head. For good reason.