Interesting. I admit I was largely going on what Mr. Harris said on the episode when responding to several posts.
replyIt can lead to hidden reasoning, if the looping allows it to stuff enough information outside visible CoT. Open AI demostrates such an ability by asking it to solve problems while thinking about something else entirely. All the other models are unable to do this except Astra. It doesn't have to be a substitute for CoT to cause monitorability issues.
replyIf you ask it not to think about something that doesn't cause the pink elephant issue?
replyThere's latent space thinking inside the model and then there's the thinking chain of thought words you see the model output. Of course the former is still happening even when you say 'don't think about it' but the latter can be controlled a great deal better with Astra.
reply