upvote
RAND & others have published work about this, and they believe frontier model training would be much easier to track than nuclear weapons were

You need a datacenter larger than a football field that draws the power of a small city to train a frontier model, and need chips that are produced by the most complicated supply chain in history (owned almost entirely by US allies like Taiwan & Europe). And that's before you consider on-chip mechanisms that seem entirely feasible. This is much easier to trace & much harder to reproduce than precision spinning containers.

Though I agree seems much harder to detect people running models that were already trained.

reply
> The only reason nuclear nonproliferation works is because the inputs are relatively rare and traceable.

There are only three companies in the world capable of mass-producing the chips used in SOTA AI training, they all use machines from a single company (ASML), and it also requires enormous power-hungry datacenters. Meanwhile, uranium can be dug up from the ground. You say that buying centrifuges will get you noticed, but why can't we do the same for SOTA ML hardware? It honestly sounds easier to control. And differently from nuclear weapons, both sides are afraid of rogue AIs.

This is a danger to our civilization, I'm sure we can figure something out.

> I have all the tools to develop my own ML training pipeline on my machine today.

Are you saying you can train the successor for Claude Opus 5.5 on your machine?

Already today you can get on a list by buying too many smoke alarms due to the americium content, or drug precursors, or a thousand harmless but suspicious products. I would be ok with a world where you likewise need a justification to buy thousands of GPUs at once.

reply
Maybe we just have very different imaginative worlds, but after seeing the kind of crazy shit that people will believe thanks to online radicalization, I'm a lot less sanguine about the possibility that people will take a shot at your list of horribles. There are a lot of people out there who seem to genuinely believe that what's at stake is the very survival of humanity and the continued existence of truth, art, beauty and all the other good things. After enough exposure to that kind of pressure-cooker radicalization, something like a repressive world government that uses violence to ensure that only the "right" people have access to dangerous things like literacy and computation could seem like a small price to pay for saving us all. Sure, it'd suck in a lot of ways, but Altman and Amodei will have left the brave revolutionaries no choice.

I think it's very unlikely something like that would succeed, but I'm increasingly sure we're going to see serious attempts to do all of the things on your list, and they could do a lot of damage even in failure.

reply