There is a decent solution for this, you essentially define your own minor mode with key bindings you want to take precedence. I haven’t tried it myself yet, but seems useful in theory https://youtu.be/D99GB591Vgo