upvote
There's already open-source stuff [1][2] out there that can lipread video to transcribe text, but the word error rate is pretty high (and it may not be physically possible to get this down without doing something like subvocal and having other sensors near your face).

1. https://github.com/mpc001/auto_avsr 2. https://github.com/amywork777/lipflow

reply
Have you never talked / mouthed without making sounds? Never even as a kid when something made you mad and you paced around the room think-saying shower thoughts to get back at that person? Even as an adult I probably mouth a sentence or two e.g. "What the fuck is this" when looking at AI output 3-5 times a week

It does feel a little weird to do it voluntarily when not emotional, but that'll be solved as soon as Claude fucks something up :)

reply