It's not intuitive to me for why preference for its own writing would emerge, and during what type of training or tuning.
replyPerhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.
It’s optimizing for good writing. Therefore, it believes its outputs are good. Therefore, it believes inputs that look like its outputs are good.
replyThis minus the word "believe". It's explainable simply by marching by similarity
replyIt would need to be researched, but I wonder if it ends up being something that happens at the token level?
reply