It's because they don't really have a mental model. RL with Verifiable Rewards can reward working code, but it can't tell you whether the prose make sense or not. In theory RLHF should be good with this. In practice, it appears to be a measure difficult to quantify and steer. I am not an insider though. So it's just a theory.

I have the same impression. It is an interesting phenomenon. Is it one of the ways we will gradually learn that AI in the form of LLMs is fundamentally lacking?

I use AI all the time, it gives a tremendous boost in speed for many tasks and it is most certainly here to stay, but there is something off about certain things, like code comments. Like a weird person.

I kind of wander if the comments are intentionally bad, providing programmers something to bike shed about. A sneakier approach to the flattery the models used to so use so heavily.