Tested your example, the confidence is 0. In smart home context, I can think of an application where the low confidence answers can be forwarded to cloud, whereas the vast majority generic queries solved locally, if the confidence is reliable enough. The response is quite fast by the way.
replyIt's not a conversational model. It's meant as a local tool calling model.
replyYes, I think OP understands that. What he and many others in this thread are trying to understand is what makes this model useful.
replyTurning a voice command into a tool call should be self evidently useful, being able to do that on a small embedded device is the novelty here. In this theoretical example, the thermostat is hosting the model on device and would use no external services.
replyI confused by the dispatch model. Tool calls typically need some reasonability to be deterministic and, more importantly, predictable in response (o/w GIGO). Why would the thermostat need to interpret a voice command that the node capturing the voice command would not interpret instead?
replyThe node capturing the voice command could be the thermostat. From my understanding they are targeting very small devices.
replySo that could be a master home automation node, but why not also a single purpose device? I can think of more bad examples than I can good ones, but maybe I am doing some soldering and I need my soldering iron turned up a bit; my hands are full, so doing that by voice would be useful enough. Something I would never link up to a big AI model or home automation network, but could be useful to control by voice.
If it's something that can be burnt directly into a chip and shipped with the products for cheap, maybe that's a more pragmatic way to get AI into small devices (see taalas for a much bigger model doing that, althoug not yet cheap).
Oh I agree that's not unreasonable. I wonder about the harnessing heft required to make it feasible though. If I say I'd like it a bit warmer, a tool can deterministically bump a few degrees while an LLM might bump it 10C. So do we limit the tool's range?
replyThat's exactly what they were testing and what it failed at
replyYes, and it seems the tool called here was the thermostat, to set the mode literally the opposite of what was requested.
reply"Make it warmer"
tool that cools room called
deleted
replyTry asking it to set the thermostat to a value. It's a very small model, there's not much reasoning capability
reply