[delayed]

I can definitely see value in that, "You know what's possible until some shows it", like world records in running and more.

But on the other hand, when I'm playing TIS-100 and similar games and get presented with a score and a leaderboard when finishing a level, I also don't want to see 100s of top LLMs, and no humans, but would much more prefer to see only humans to compare against, as I don't regularly "compete" against LLMs, and wouldn't see much point in that.

But yeah, online leaderboards are definitely dead now, unless we figure out some foolproof human attribution mechanism, which feels kind of further away than ever.