> they frequently generate CoT explanations rationalizing those answers

I can't tell if this is a really human thing to do, or if it's just a sycophantic thing to do when your master likes a good story.

Fair, but you can see the logprobs, right? That alone is pretty valuable.

This is like one of those folk theories that have some truth but in practice it is useless.

I love COT and gives me a good idea about the thought process. Sure it doesn’t work always

In practice, it does help to immediately be able to tell if you missed something or wrote something incorrect in your prompt, compared to if you didn't have access to the CoT. If you see that it goes off wondering about stuff or thinking about stuff that you know are already solved/known, you cancel the execution, edit the prompt and redo, then observe the CoT again, until you've resolved any "misdirections".

Both OpenAI and Anthropic hides so much of that, that you don't really know if it misunderstood something or you got something wrong, until all the turns are complete or you directly see tool calls that implicitly tells you something is wrong.

You don't need to read any papers to see this effect concretely when developing together with LLMs.

[dead]