The solution is to not wear them as long, especially right after a shower or the like, when there is extra moisture in your ear.
Alternatively you can put alcohol in your ears to kill the bacteria, sort of like you’d do with mouthwash. Any pharmacy will sell over the counter ear drops with alcohol, meant for drying out swimmers ear. You’d have to keep doing it once every week or two, but it would keep the infection away.