upvote
I wonder if Google regrets publishing that article on transformers.
reply
Urs used to talk (internally) about not publishing "industry-enabling papers" which is why most Google infrastructure papers were describing something that had already been turned off, or was already in the process of being replaced by the next system (GFS, Vitess, etc). The things that did get published were either considered not key advantages, that other companies simply cannot do, things that other companies wouldn't bother doing, or experiments that never worked at all. There were exceptions of course. But it led to a public perception of the Google stack involving mostly technologies that were long dead or were never adopted.

"Attention Is All You Need" was a very very different thing and I also wonder if they are glad they published it. But I imagine if they hadn't, the motivation for researchers to leave Google would have been even larger.

reply
> I also wonder if they are glad they published it

https://youtu.be/ue9MWfvMylE

Jeff Dean is asked this question by Geoffrey Hinton at 37:35 - might worth watching. Overall an interesting video.

reply
So Google allowed publishing the Attention paper because they didn't understand its value.
reply
They patented it. When the dumb money stops sloshing around, we'll start to see the fallout from that.
reply
Why do you think Google considers Anthropic a competitor?
reply
It makes every bit as much sense as investing in Snap while still operating their own social network product. Seems to have worked out fine (for Google, not Snap).
reply
FWIW I’d buy SNAP now that they are at rock bottom
reply
Given that anthropic is probably paying it all back to them in compute bills, they may not be giving them anything.
reply
hedge your bets, I know I would
reply