upvote
The thinking process is helpful, but it's not strictly predictive of an LLMs output, and its not really the model's "thinking", although it clearly has an effect and a benefit. I cited one paper on the topic, Turpin et al. So I'd be wary of reading the thinking tokens and inferring they tell you how a model decided something. And an LLM can output things you never asked for, including dangerous things. With a decision model, you can just constrain its choices to those you're comfortable with it making. An LLM requires another layer of interpretation of its outputs to get the same property.
reply
I agree very much, not blindly trusting model output is Rule #2 for interacting with Ai

I use the LLM coin flip as a comparison point. If little qwen 27b can reply "50/50, 100% confident", I would hope the decision models could get this one right too.

reply