If they were benchmaxxing, surely they would score higher than 30% on ARC-AGI.
replyDoing a quick search it seems like the average human score is 49%?
replyI view benchmaxxing as more of a spectrum. Mmaybe they're doing a lot more RL in environments similar to ARC-AGI 3, not even with the purpose of scoring well on any benchmark but hoping it generalizes into better performance on real, useful tasks.
nah they could make educated guess about arc and benchmaxx it too.
reply