Interesting how do you know if it does a good job at these tasks? It seems that thinking in LLM's has been one of the things that has been so important to making sure it's a good response.
replyIt doesn't. Decision models are not more accurate than small models with no thinking. If your classification task is accuracy sensitive, you still need to stick with one of the big frontier models.
replyIf your classification task is accuracy sensitive, you still need to stick with humans
reply> If your classification task is accuracy sensitive
replyWhich classification tasks aren't?
Support ticket routing to a team. If the model gets it wrong, the team can put it back in the queue for manual routing.
replyAnd if the model gets it wrong in the other direction, well, you didn't really need that customer anyway.
replyWhen the use case allows room for an acceptable amount of incorrect decisions because there's a manual flow those cases can be routed to still anyways.
replyMy problem is no other tokens means I can’t get reasoning or a justification.
replyIt can be used a fast/cheap router, if a response is above a specified probability then send it to LLM based workflow branch, otherwise nothing more to do
reply