If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters.
You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US...
Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects.
Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.
Since many people haven't seen 10MW cabling for a data center or how a big GPU server is cabled, they naturally imagine connecting servers is akin to plugging an appliance to a wall.
When the electricity provider says "I neither have the capacity, nor the required cables in that area", thing gets real.
The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.
See PUE (Power Usage Effectiveness) for its scientific form.
love to debate actual discission points. pull up "datacenter dfw" on google maps for mine.
https://youtu.be/_bP80DEAbuo?is=sg09k66iutKFIFSo
Yet here we are, discussing "data center" as if they're standardized and of similar (nose) isolation.
There are no meaningful regulations in building them, and they can be incredibly polluting. So your experience with a potentially well isolated one is sadly not the norm going forward. And we don't even know how close you lived, if you're eg talking about "within 5km/3miles" then your experience would also have little value in this discussion in general.
Can you cite a source for this? It's not in the video, as far as I can tell.
I would be wary of Benn Jordan's videos. They are full of mistakes and misrepresentations, as Andy Masley has convincingly demonstrated: https://blog.andymasley.com/p/contra-benn-jordan-data-center...
I recall seeing Benn Jordan's responses on Bluesky and thinking they were quite poor. He was unwilling to admit to mistakes, and kept trying to grasp at newly searched papers that didn't actually support his arguments.
Indeed, he shot himself in the foot there pretty bad, but I would argue that that was just the result of successful Agitation.
I would personally strongly prefer being in the same room with Benn compared with Andy, because one of them is authentic, while the other is calculating. Though, arguably, Benn has been catching up on that lately too.
But yeah, taking stuff with a grain of salt should be the default regardless of the person speaking.
10 years ago, I was running 4 CPU servers with 48 cores and 128GB of RAM in 2U enclosures with a maximum power consumption of 500W or so. I was able to stick ~20 of them in a 42U rack, totaling 10kW.
A data center full of these can be cooled with CRACs and hot/cold aisles without much problem. This is still too much for a bog-standard server colocation operation, but for HPC, that was normal and manageable.
Now, a ~1U server houses 4 SOTA NVIDIA GPUs, 64 cores, magnitudes more RAM. This server alone uses ~3KW of power. This means you go anywhere between 30kW to 50kW per rack, and you have many racks.
Of course this means more power comes in, more heat comes out. This means more sophisticated infrastructure: bigger and beefier primary and secondary power systems, beefier cooling, more heat, more noise, in short "more of everything".
Of course when you cram this much energy and heat into a relatively small space, its effect on the environment will be much more pronounced.
Facebook's previous SOTA datacenter used water infused, HEPA filtered free flowing air accross the datacenter. Now, it's server level direct liquid cooling with extensive water treatment and oversight on coolant parameters.
Compare this having a hand warmer vs. coal ember in your hand. The latter needs a much more elaborate setup to prevent it burning you badly.
You can stuff GPU servers into existing buildings- but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.
1. Article is about AI, so I have given the example for an AI datacenter.
2. In pure CPU datacenters, the power dynamics do not change much. I can add more servers to a single rack, but the rack power is again in the 30kW to 50kW range, so you're planning and building for the same power capacity.
> You can stuff GPU servers into existing buildings-
Yes.
> but even with significant upgrades you end up with a lot of empty space on the floor that can't be used.
Yes & No. It's not impossible to convert an old datacenter to support ~35KW/rack capacity, but it's not cheap, and you'll have more worries than holes, piping, building and power. Namely, can your floor handle that much weight to begin with?
If we had regulations on noise, vibration, emissions, water use, electromagnetic radiation, whatever else, then it wouldn’t matter what people tried to build — if it fits within the guidelines great, otherwise back to the drawing board.
Putting “data center” in your ordinances is as lazy and ineffective as putting “abattoir.”
We certainly do! It’s just often overridden and ignored for these companies and data centers
Sane jurisdictions do have regulations regarding these things. Not all jurisdictions are sane, some of them are run by people who sell out their residents.
Suburbs and cities around me all have noise regulations, my state has its own pollution regulations, and the local water utilities don’t hook up customers that stress the system. Unfortunately there are places like Texas, Tennessee, Louisiana, Mississippi that don’t give two shits about their citizens and let companies run temporary natural gas turbines permanently and all kinds of other nonsense.
This does sound plausible, but it's also pretty sad and not a sign of a healthy democracy
If a municipality doesn’t have emissions, noise, water use, etc regulations, that’s a serious failure in governance.
We don’t need nor want the word “data center” in regulations anymore than we need the word “abattoir.”
The names of the things we build change all the time. Their impact on their communities don’t.
We need to regulate impact, not the name or type of business.
If we did, nobody would know or care about data centers and they wouldn’t be affecting their communities, because they’d be operating under established impact regulations.
Would love more calculations on that