This is speculation, but funnily enough Gemini models show the least evidence of being distilled from OAI or Anthropic compared to everything else in the market.
replyI feel this is correct. Google has probably the most data of any company in the world. For better or worse, they don't need to train on the outputs of other models.
replyAt a org like Google Cloud barring rogue L5/6/7s acting on their own any sniffing of distillation will get leaked. Too many chaos exist at a org size of Google to systematically do it given the contract at hand. This is a company where every spicy memo or mild gossip will risk leaks.
replyGCP already hosts Anthropic models so probably trusts and lawyers.
replyProbably a very strict contract, and letting OpenAI read all the session logs like they do for their own customers.
replyNvidia GPUs let you load encrypted weights onto them and to decrypt on chip so Google would need to have some sort of Nvidia management firmware attack to read from HBM while the GPU is live
replyIsn't distillation more of a black box type affair?
replyDistillation does not require access to weights.
replyDoesn’t really matter what happens at the GPU level when Google can see all API traffic.
replyExcept google cloud doesn't use Nvidia GPUs but TPUs.
replyGoogle has both. Afaik we don't know whether OpenAI models will run on TPUs or GPUs here.
replyI bet it would be the NVIDIA, at least initially, not for the encryption capability per se, but for simple fact that adapting to TPU at high performance is non-trivial.
replyFirst answer this: How are they trusting OpenRouter not to distill them?
replyOpen router doesn't host the models, so they wouldn't have access to the weights or reasoning traces right?
replyThey can still collect (input,output) pairs.
replyAmerica is a high-trust society.
replyJapan is a high-trust society. America is the opposite.
replyThe US is medium trust.
replyEverything is relative, but fair enough.
replyMore like - America is a high-litigation society.
replyI don't believe threat of litigation alone explains the extent of it.
replyThe way an American company, esp a reputable one like Google, conducts itself internally is different. There will be much more internal backlash to do the wrong thing way before litigation.
Contracts and lawyers are more powerful than any technical tool.
replyYeah like the copyright lawyers of content owners that got their works trained on?
replyHow many trillions of dollars were those content owners worth?
replyEnough. They just weren’t working in coordination.
replyYou are vastly overestimating the power of publishers. The top book publisher in the US is worth $5 billion. All publishers together are about $50 billion. The New York Times is $11 billion. Fox is about $15-25 billion. Add together every single book publisher, newspaper, magazine, TV/radio, digital publisher and really every print and broadcasting platform in the country and you’d reach about $100-$150 billion. That is roughly 3% of Google.
replyThese companies have a ton of cultural capital, sure, but in American capitalism that means nothing.
Google Cloud couldn't give a shit about Google Deepmind, and backdooring Clouds customers only stands to hurt Cloud.
reply