> I don't understand why you can't have any protocol that can (potentially recursively) resolve to a layer 2 address as a next hop.

Which Layer 2? Ethernet? Token Ring? Infiniband? ATM? Fibre Channel?

Both IPv4 and IPv6 were created when there were multiple Layer 2 protocols in use. That's why where was a completely different protocol to handle Layer 2 to 3 mappings (often itself referred to as a Layer "2.5" one):

* https://en.wikipedia.org/wiki/Address_Resolution_Protocol

That's basically MP-BGP! Your dream is real.

(Keep in mind that the setup described here has work for years now.)