Maybe I'm misunderstanding you, but it's not necessarily a terminal feature per se. Have you tried setting `vi` mode in your shell? There is a bit of lag going into normal mode in ghostty; maybe ghostty is configured by default to wait for a second keypress when you hit Esc and you can disable that.

Bash (also works in zsh): https://www.gnu.org/software/bash/manual/html_node/Readline-...