upvote
>The operational engineering that goes into serving a frontier model is _highly_ nontrivial, it is not commodifiable. This isn’t a SPA application.

I don't really see it frankly. Admittedly I never tried it (I only played a bit with local models) but conceptually how is it much harder than all the other cloud services that already exist? My educated guess is that building a competitor to Netflix, Youtube or Dropbox seem technically a lot harder to get running than offering inference on demand for instance.

It seems that the main barrier for competing in that market at the moment is getting your hands on the GPUs/RAM and a datacenter to put them in.

reply
> Similar to how “the Chinese” haven’t undercut on cloud providers’ CPU offerings

No Western company will be allowed to choose a Chinese cloud provider by their legal department or by their stakeholders. Which leaves only the US cloud providers as viable sources (as us, Europeans, have been too busy building our revenues based on turism and other such non-tech thingies instead of building real stuff).

reply