undefined

points

[-]

Inference workloads are usually a lot less picky about the exact hardware than model training. At least in the cases I know of the models are trained on Nvidia hardware, then exported and run on a mix of Nvidia and AMD

by minraws4 hours ago|

parent|

[-]

At scale for inference it's almost non-existent for a data center company to go for AMD because they couldn't get or afford Nvidia atm.

They instead start the build out and plug in stuff they can, then take a loan or ask Nvidia to help fund it. (I am not joking)

I believe the case is if you can prove to Nvidia you can install and provide more Nvidia capacity they help out because more Capacity going online today is in the best interest of Nvidia.

Spot prices of Nvidia GPUs going up is not good news for Nvidia btw. The people renting Nvidia has the least amount of friction in moving off Nvidia, especially with AI tools you could build and get up to speed with AMD stack much sooner...

So if Nvidia is truly not an option and you entire company is not a bet on Nvidia then you will move off but only as a renter not as a buyer unless they truly can't fund Nvidia I suppose.

But again I repeat if you build a datacenter and provide good enough base Nvidia will help fund you to a mostly complete data center.

People might not like it but that's the reason Nvidia is so unreasonably dominant even now when otherwise given the scale of investments it might have been cheaper to look for alternatives.

This is why Nvidia doesn't like the China stack.