Malta has a population of 500k. Let's assume 100k people use MaltaGPT daily, and they send an average of 10 messages per day, so roughly 1M messages per day. That averages 694 per minute, but at peak could be 3-5x that, so let's say 3000 per minute. Usage will of course vary by day of week and time of day (they could partner with a Pacific island and share inference hardware).

Those 3000 messages per minute translate to 50 messages per second. Let's say average prompt input is 5k tokens, and output is 500. So 250k tokens per second for prompt processing (let's ignore caching for simplicity) and 25k tokens per second for output decode.

If we take a 500B dense model, that concerts to roughly 1 trillion flops per token. So we need 250 petaflops per second of prompt processing and 25 petaflops for output decode. So 275 PFLOPS in compute.

That may sound like a lot, however a NVIDIA DGX B200 machine (8xB200) has a compute of 144 PFLOPS at FP4. That is assuming 100% efficiency which isn't really possible, and we also need to factor in memory usage which we would be limited more by than compute. So let's say we'd need 10 of them. For an entire country to have a sovereign version of ChatGPT.

The cloud cost to rent one machine is around $50/hour, so that would mean our cluster comes to $4.8m per year. However the list price of a machine is around €400k, so the price to buy the cluster outright would be around €5m (you need the rest of the data center too), with operating costs of around €500k per year.

So per citizen: €10 upfront and €1 per year.