You're right that having a vanity domain for your primary email address isn't for the faint of heart. There isn't any realistic advice for the average person because it's not for the average person.
Sure, you'll likely miss some emails, but otherwise it's safe.