My experience is that Emacs wants to be your everything, and works best if you let it. Have you tried putting your terminals inside of Emacs instead of the other way around?

* If you need a "real" terminal emulator, you can use something like vterm (https://github.com/akermu/emacs-libvterm/).

* If you need to be able to attach/detach Emacs sessions on remote machines, you can use something like dtach or abducto (https://www.brain-dump.org/projects/abduco/).

Tmux conflicted with my vim bindings and i tweaked the tmux.conf to fit my needs. It's worth playing around on pure tmux and figuring out what would work for you.