Context is useful. The parent said: "it's a cheaper product that's almost as good or better in some cases"
replyThe only open models that are "almost as good or better in some cases" require massive amounts of RAM. I posit that most people cannot afford a decked out Mac Studio, and therefore run the smaller "flash" variants on more normal devices. The issue is that those are nowhere near frontier-level in terms of capability.
"Running your own infra" also includes managed infra like Bedrock, Foundry, etc.
replyNot just your local machines.
Enterprises are where you see this adoption. Legal, finance, tax; sensitive context where the data must be contractually opaque to external parties.
Precisely. I have figured out a nice recipe that is quite affordable, 288G of VRAM for a little under 20K, it takes some fiddling though, but once it works it is really neat.
replyPCIe is incredibly powerful tech.