how the hell do we even track that? and before someone says...
reply—"Benchmarks!"
...I'll tell that they can be gamed so easily, and they are on a consistent basis.
Sure they are, but do you think they are continuing training to improve a model after release without bumping the version number, presumably only to game the benchmarks?
reply