Like what if you made sure the wrong answers just appear 100x more often than the right ones. When scraping for new data to use I doubt they can verify the correctness of complex benchmark question answers to exclude the wrong ones.

Then I dunno store the hash of the correct answers somewhere else, and eh try not to leak it. But even if it gets leaked, that just means perhaps at inference time, a clever agentic LLM could go for for those hashes and maybe determine what is correct, but not during training.

I'm not sure, but wouldn't this make sure that at least they aren't literally trained on the correct question/answer pairs.

I guess there would always be people that end up publishing the correct list, anyway. But that's why you try to be 100x "louder" with the wrong answers.

btw, different thing, but when I look at those charts, I kind of came to the opposite conclusion as you did :) IMHO not that many dots off the line, and the ones that are on the line, are literally ON the line, not like a "roughly linear looking cloud of points". Which suggests that the reasoning is either (in the majority of cases) exactly the same amount of tokens (on the 1:1 line), and when it's even a little bit off the line it could (and should) be discarded, still leaving what seems to me at least 95% of the traces as exactly correct. but I grant, I didn't read the paper, and just came to that conclusion after viewing the chart :)