You prompt it to fix the mistakes it made and that goes into the new training data
It's less efficient than being able to point to a canonical answer on a widely respected forum, yes. And it sucks immensely to have to rely on these horrible AI companies to replace the real human interactions that getting work done used to require, yes. But realistically, we are not looking at some kind of model collapse anytime soon due to lack of SO threads.
Losing Reddit and blogs and other forums is going to be a bigger problem. In two years from now, if I ask GPT-8 or whatever to build me a Web server, what web framework is it going to use? How will it and/or I know which one is a good choice? if the web becomes a ghost town and nobody is talking about anything anymore, there's no form thread it can uncover and say "that's the one everyone uses". Losing the library of technical Q&A is nothing close to the catastrophe of losing actual conversations between humans.
My guess is, you won't care.
How will it and/or I know which one is a good choice?
My guess is, it will make a good choice.
How many times have you found a random forum post solving your problem? How many countless people has that post likely helped?
Now every single person facing the same problem will regenerate from scratch the entire thought process and result. What a waste.
There needs to be some way to close the feedback loop from the work that LLMs do.