NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost.
replyOne objective of the project might be simply to provide credible negotiating leverage when dealing with existing suppliers like NVidia. You don't have to deploy at scale for that to work, but you do have to look like you could if pushed hard enough.
> NVidia has enormous operating margins, so a competitive solution doesn't have to match or beat NVidia's scale efficiencies; it just has to beat delivered cost.
replyBut then that means you have no actual moat against the behemot, right? Your competitor can move into the market as soon as they want to, at much better cost (so at slightly better price)... and Nvidia certainly can adapt much faster around hard hardware specs innovation than a new entrant ever could.
Those are not OpenAI's concerns - they just need to scare NVidia enough to lower their prices more than they'd otherwise want.
replyBut they are also PURCHASING from nvidia so any time nvidia lowers their prices they save money.
replyIn the short and medium term, it probably won't be more economical to produce for OpenAI. Where OpenAI is benefitting from their own chip is being able to tailor it to their models and workloads. When you buy off the shelf Nvidia, its not perfectly tailored and OpenAI has to spend marginally more to run off that chip. At the scale OpenAI is operating at and plans to operate at, that margin becomes pretty big $$
replyBecause it's actually Broadcom that is doing most of the work.
replyBy leveraging the experience Broadcom has in this area. Still remains to be seen how that goes when they want to scale production.
replyReplace OpenAI with Apple and Nvidia with Intel.
reply"How can OpenAI produce a LLM at scale more economically than Google, Amazon, and Microsoft which have experience in planet scale software and scale efficiencies unlike them".
replyOne answer is they're quite good at poaching talent.