Considering thats not the case, what you just did is move the goal post to a account recovery process. Question becomes who has ability to recover the account, in case its tied with email then most likely it has to be a shared email box. What you have now is a much more fragile system in case of custom domains, where whoever is controlling the email domain (DNS management capability) can take over the AWS accounts.
An email per account where only security team has access. Whoever can modify domain can already do this.