upvote
We're like 3.5 years into this new era - I'm not counting winners or losers yet.
reply
How is it far behind? The benchmarks published in the blog post show it is superior to Opus 5.5 and Astra 6?

Behind how?

reply
Within one question of their web interface, it has lost context and asks you to clarify what you are talking about.

I am very often giving the same programming task to multiple LLMs for various reasons - the answers from Google are so bad that I gave up.

I have no interest in benchmarks.

reply
So you have no experience of their latest model release then? Just repeating the usual tropes about Google having messed up? Or basing your opinions on their website chatbot?

If you have actual independent benchmarks and evidence about how this new model release is "so far behind" and refutes the stuff from their blog then please do share because I think we'd all love to see that?

reply
No I am commenting on my real world experience of using Gemini daily. I still ask it questions alongside Claude and OpenAI and Gemini is always the worst of the three.
reply
So you've not used this new release then? So how can you say that they are "so far behind" if you are not using the most recent model for your comparison. This is their first 4.0 model, that you are not using and instead basing all your opinions on on some ancient months-old model from a previous generation?

With respect, I don't find your arguement about them being "so far behind" especially convincing when you are using previous-gen releases and not actually using their current release.

reply
Yeah, they've definitely got some recurring tooling/infrastructure problems around the models.
reply
deleted
reply
Funny how we start to see people supporting LLMs like we support sport teams, political parties, or celebrities.

- Person 1: X is garbage compared to Y!

- Person 2: Why?

- Person 1: Because I like Y.

reply
I wonder if Google bans internal use of Claude/Codex.

And I wonder if Google's main monorepo is already in Anthropic/OpenAI training data because of some stubborn dev.

reply
(I work at Google) Yes, internally we all use Jetski (internal version of Antigravity). Outside of Gemini, Opus models are supported and allowed for internal use. No OpenAI models since they are not on Vertex
reply
No way to run OAI on a machine with monorepo access even if you wanted to. Claude runs on Vertex so it's not leaving Google infrastructure.
reply
Claude used to be GDM only, but recently opened up Opus for all googlers
reply
I've tasted Gemini through an intermediary and it feels far better at attention to detail than other models I've tested (Claude Opus/Sonnet, GPT whatever it's called nowadays). But it's less likely to get one-shots right.
reply
They are playing a longer-term and more enterprise-oriented game.
reply
deleted
reply
Google's strategy is to let their competitors bankrupt themselves while they continue to offer good-enough models near breakeven.
reply
> Gemini is so far behind that it is effectively useless compared to Claude.

I fundamentally don't understand LLM "brand loyalty".

All of the models are constantly leapfrogging each other and always have been.

Google had a long lag between releases (and still hasn't released Argon), but why wouldn't they be able to compete? It isn't like any of this stuff requires secret knowledge, the Bitter Lesson has proved true again and again, and Google can certainly scale computation, it is like the one single thing they've always done well in spite of all their other foibles.

reply
Its not brand loyalty. I use them all the time and have no loyalty - I'd happily ditch an LLM for better results - that's how I got to Claude from ChatGPT.
reply
Gemini has never ever leapfrogged any competitor.
reply
I hope it can. Today it is very far behind.
reply
-
reply
You have access to Argon?
reply
Google employees do.
reply
Their profile says: > Currently at Google as a Sr. SWE SRE on the cloud.
reply
deleted
reply