upvote
I think you'd need to insert "critique these ideas." That's where the intellectual honesty comes in - I suspect that _all_ current LLMs would continue on with and truly stupid ideas as though they were gospel, their profound incompetence at backtracking on what they have said is something the industry hasn't figured out.

If we could make progress in that area, maybe CoT could gradually decrease as it approaches its limit, or maybe the LLM could control the temperature of the next token itself (how this would be trained, I have no idea).

reply