It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content.
They also have very good code and playbooks for actually doing a fine-tune, CPT, etc.
Even if you’re not tuning a Nemotron model, its mixes are very excellent for your replay data slice; or general experiments. Way better curation and quality than Dolma, etc; or other large huggingface data mixes I tested.
There's a large market, very large, who want the best regardless of what it costs. Probably a large enough market to keep that domain of research afloat (as opposed to shifting research manpower to cost cutting).
The reasoning is just that the marginal cost of AI is very secondary to fixed costs of the businesses themselves; it's not an excuse to sacrifice performance.
Claude models weren't really good or noteworthy until the 3 series anyway.
All of the Gemini models have been considerably behind the capabilities frontier. The only exception was 3.0 which seemed quite good, but had latent issues and we were all measuring with the incorrect metric, agentic where it's latent issues were very pronounced.
GDM+Google may have created an exceptionally efficient LLM for serving search. This is likely a great accomplishment (or maybe Google is burning money at a rate unheard of before). But Frontier capability: they have never been in the race.
This is sad, since they had everything necessary to be on or beyond the frontier.