Arguably, missing IP literal domain support is pretty far down the list of things creating today's actual mail delivery walled garden.

Wasn't this removed from the email standards?

Most people don't own their IP range so it's moot point to support it.

It is not a conspiracy theory that it is hard to maintain reliable delivery with self-hosting email. It is primarily because email filters are in part based on trust relationships, and huge amounts of spam come (or at least, did) from relatively unknown originating servers.