this feels like a small question, the money needed for this isn't that much (probably ~20b) and every country needs cyberwarfare abilities, especially automatable ones, so the military can pay for it. And its not like 10T models cant be used for anything else, so people who want ROI in health, chip development, all sorts of stuff
I can give you one concrete example. Google for Stuttgart 21. It’s a project to build a main train station in Stuttgart, Germany. The project has been delaying for years. By the time it’s finished it would be 12.5 years overdue its original deadline and so far it has costed 14.5 Billion euros.
One could assume that the biggest economy in Europe can build a train station in no time because how hard could it be? Given the fact that they can’t even do that, I can imagine how it would go with publicity funded cutting edge technological AI endeavour.
Provided that the systems work, they can at least be repurposed to other things. If the organizations that are to train public LLMs can't do it, we can rent the system out to Mistral or something.
So I think something like 5B, starting with 10B to get started, in public money per year, the chip firms are private, some of the LLM firms will be private, but the system is available to train European LLMs-- that's I think a realistic approach.
That’s why I think it’s not going to be successful with public funding. What EU needs is structural unpopular reforms. Reduce taxes so companies and top talent would have an incentive to choose EU over US. Reform employment laws so people can be fired for being pencil pushers and lacking off. Relax privacy and copyright laws so the data can be used to train the model. Completely repel all laws that create a bureaucratic nightmare for startups. Today all these suggestions would be a complete no go in any European country, but that’s the hard reform EU needs like yesterday even to have a chance to compete against US and China.
I think there's nothing special about public funding though. The field is so competitive that people will be mad if a competitive model is not achieved, making corruption more damaging to the organizations. There would also be some internal competition. There are after all several EU LLM/AI/etc. firms that would probably try to use this infrastructure.
I guarantee you that maybe people would get mad that no competitive model is achieved because the politicians burned the tax payers money. But there would be no repercussions, nobody would be held accountable and would just be marked as a failed project.
Some sort of alternative must be created, after all, since models are being restricted to the US only.
But no, lets make this a 10B deal who someone will suck out 1B out of it even before its shipped
pretraining becomes more expensive actually as you make MoE models sparser (you need more tokens in the pretrain, and if you don't have that then you need to train for longer)
If we end up with a world where only US firms can use the latest LLMs and the latest LLMs are needed to keep up in the software world, or in making prototypes, then that's a whole series of fields which are blocked from us.
So I think we need to make sure that we no only can, but build frontier LLMs from scratch-- not RLed on foreign data, but genuinely from scratch.
Even from an economic point of view, I don't think a continuous outflow of ~200 USD/month for every office job is sustainable, and that's what we'd get if the plausible scenario is borne out. An inter-EU cost of 200 USD/month for every office job though, that's survivable.
also an em-dash and genuinely in the same sentence? cmon bro how much claude have you been using?
With regard to the substance: sales of ASML machines are not currently connected to the EU getting chips, but to ASML getting paid for their work. For EU chips for training transformer models we'll need chip design firms, not chipmaking, and as I stated in my comment, there are some promising ones that will probably be able to design the chips we need if we order them.