That is an effect but it’s not a nail in the coffin. There are lots of proprietary benchmarks on real product traffic that aren’t contaminated and open questions as well. People at these labs largely know what they are doing, it’s not like people don’t know this.

Is this not true of human intelligence as well? Many smart people I know hold beliefs that have no obvious truth value.