I haven't tried it, but this looked promising for that exact task: https://huggingface.co/superwhisper/s1-mini

I think the model can even evaluate itself. If it looks afterward at an output like "do you. Want to get lunch?" in the absence of affirmative evidence that the user wanted it that way, it should be able to see that it goofed.