I do think it's very likely that OpenAI pays for solutions like these to put in the training set, and then we get material like this Reddit thread. They market themselves as selling "intelligence", and solving these math problems is something people view as highly intelligent. I'm not a mathematician, so I cannot fully judge it, but based on my experience using LLMs for novel problems in other domains, they seem to really struggle with things that aren't common. That leads me to believe they train for specific outcomes like this. Also, there are a lot of jobs out there for data annotation, including software problems (Meta has basically reorganized its entire engineering department to create training data for coding problems).
This comment on the Pelican svg better articulates what I'm getting at: https://news.ycombinator.com/item?id=48950883
The way you should read this is (IMO) not that LLMs have somehow achieved AGI, but that a lot of mathematical research is more about knowing a huge amount of mathematical background, being stubborn, and getting lucky with an approach than it is about brilliant insight. Many people who don't think of themselves as particularly mathematically gifted could have made progress on these problems if they were given enough time and were interested enough. What's notably different about 5.6 (and born out in benchmark after benchmark) is that it does seem to genuinely "reason" through stuff at all -- without that, persistence is pretty worthless because the LLM just goes wildly off the rails if it's put to work for long enough (5.6 itself will still do this if it can't find an answer in a reasonable amount of time).