I am not colorblind, and have the same experience. It's made worse by the fact that different apps use colors differently, so if I find a theme that works well for some clis and tuis, there will be others that will have unreadably low contrast with the default theme. So I have to tweak the colorscheme for each application as well.