If you have your own custom domain, changing providers is pretty easy, IMO. In fact, the portability factor is probably the best benefit of having your own domain.
Then I went through that list every now and then, unsubscribing or changing email addresses as they came up. At the moment it's mostly just mail from sites I've used Google OAuth for some reason and can't be bothered to change (or just plain can't - one site actually uses email as a primary key and are completely 100% unable to change it.)
That does indeed sound as daunting as I feared, unfortunately.
You don't have to do it all in one go. It isn't like you have to move everything all at once like you're moving to a new apartment.
I started moving away from gmail years ago. It took probably like 5-10 minutes to set up email under my own DNS control and to set a forwarding address on gmail that pointed to that email.
I didn't even bother to do an initial bulk move of things over to the new email, I just switched things over individually as needed in a sort of on-demand fashion. 30-60 seconds here and there moving over individual accounts one at a time and only when it became obvious that I was moving over something worth keeping up on.
Spread out like that it felt like almost no effort at all.
It's probably not even several hours of work for me, except that I do want to set that time aside in case something goes wrong and I need to revert things; I can't afford to not receive email anymore.
That, and I do want to transfer all my existing email history.
Every time google did something to piss me off I moved an account or two in anger. It felt good. It felt so good that when I searched my gmail and only found google-related hits in my password manager, I was slightly disappointed lol
(Don't get me wrong, I still want to do it, but it's still daunting.)
The only thing that's a bit of a hassle is the calendar. It also mostly works, since you can add your Fastmail calendar to Google Calendar, but Google is slow to sync.
I have no cloud document, email or photo storage at all. Just an IMAP box.
This is one of the big reasons I switched to GMail when it was new. Most free alternatives like Yahoo would lock forwarding behind a paywall. A very transparent attempt to keep users by force.
I would receive some messages twice for some time, as people resent their mails to the new address
Things sorted themselves out without much effort. But indeed, it took at least a year or two.