No, newer models talk to themselves in this degenerated form of English all the time. It's a known problem that deep frying transformers with RL reduces CoT legibility, and the only known way around it is the Most Forbidden Technique - grading chains of thought.
replyWhy is that not only forbidden, but also most forbidden?
replyThere's Most Forbiddenish and there's Most Forbiddener. We already apply length penalties to CoT in RL, for instance.
replySometimes I worry that the framing around CoT grading prevents research into necessary and useful safety techniques that edge up against it, but what do I know I'm just a guy, c'mon n downvote me LW bros
It constantly leaks into actual writing. I periodically have to correct Astra to fix lack of spaces, and sometimes, even outright dropping of intermediate tokens.
replyBoth Astra and Sol 6.1 do it, and Sol is even worse as it leaks through everywhere. Whenever it issues a subagent prompt it's just a total vomit of tokens mixed up together.
replyI sometimes find Sol 6.1 both printing random words, and producing code that is not super-dense, but far off from normal code with no empty lines and very logically dense.
replyIt almost feels like its trying to protect its code from scrutiny, or like a developer who want job safety.