> What I'm questioning here is that there are labs who have sat down and deliberately tested and tweaked the performance for this particular task, independent of general model improvements.

Given the massive delta easily reproducible with some models, is it really doubtful that certain labs have not: https://news.ycombinator.com/item?id=48951229

We are going from pretty good pelican to jumbled mess with a similarly silly, but different prompt across multiple models from multiple labs, both Western and Eastern, both Open Weight and Closed.