Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost. This is a bad comparison.
[1] https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...
How so? First, kernel writing (or ML engineering more broadly) is a highly specialised task. Not everyone can do it. It shows that models are getting better and better at (easily verifiable) hard tasks. And you can "hire" that expertise much easier than you can hire the equivalent meatbags. And more importantly you can "fire" them as soon as the task is done. And then hire them 3 months later, when the new model drops. And so on.
Second, 20% gains in inference today gives better end results (i.e. lower overall cost) than 1% in training 2 years ago. Today's models are improving mostly via RL. And RL is highly dependant on fast inference (you want many rollouts for each training scenario). Same for dataset filtering, environment generation, distillation, etc.
> reduced inference cost by 20 percent saving company x billion dollars per month
I have no doubt that further work was required to enable this, but it's still very cool to be possible to say that.