undefined

points

[-]

This looks like a good benchmark. Time and time again I keep giving OpenAI models the chance to win me back, but Opus (and Fable especially) just writes more elegant code and is a significantly more productive rubber duck for interactive discussions. I feel vindicated seeing your description of verbose and defensive code, and I’m a bit disappointed that 5.6 Sol’s solution is still >5x longer than the human solution and 2x as verbose as Fable’s. Do you have any insight whether any of that is comments?

I wonder why nobody has tried to optimize for actual code size or complexity metric, or at least why I haven’t seen more benchmarks that display this. GPT5.5 just keeps pushing more and more pointless indirection into every function it writes in my main project, it’s borderline negative productivity.

P.S. I’d be curious to see Cursor’s composer models in there, they seem to be among the best performing low cost models: https://artificialanalysis.ai/articles/cursor-composer-2-5-c...

by desterothx5 hours ago|

parent|

[-]

Really almost all benchmarks I look at have a cost per task column, which is basically the code size metric if you take an extra step

by dgfl3 hours ago|

parent|

[-]

Not at all. The model could (and sometimes should) burn all the money it wants, and then produce a single line of actual production code. Only some things, e.g. full rewrites, have clear cost - LoC scaling.

For my usage, I would very much prefer if those $/task were being spent in thinking and experimenting, and the actual output would be as short and maintainable as possible. “maintainability” is a vague target of course, but it’s at least somewhat correlated with code size.

by CuriouslyC7 hours ago|

prev|

[-]

Seems quite kind to Gemini models.