I don't like it much, but I understand. It's the only way to be objective. Models are trained and benchmaxxed on any popular public benchmarks.

i was wondering why the pareto line moved so much. i did not remember it being all openai models. good to know i dont need crazy pills yet.

we need to make semi-educated decisions to pick model somehow, and AA is the best what is available

You'd prefer public questions and answer sheets that anyone can train on?