I've heard about rogue keyboard firmware, but that requires having a programmable/updatable firmware keyboard in the first place. And that closes the loop of my argument: People that want to update the firmware in their keyboard will do so, whether it's in the browser or by installing a potentially shady and not at all sandboxed third party application.
At least in the browser, permissions are time limited and scoped to explicitly granted devices.
> WebUSB IIRC still does not have origin restrictions
How would you even enforce these on the open web?