For .pkg files, there's UninstallPKG which reads the package manifest and properly uninstalls it.
Speaking of packages, even more embarrassing, Microsoft Windows literally beat them to shipping a first-party package manager. I feel like Apple lives in a fantasy land that the drag’n’drop app install method from the classic macOS is some kind of platonic ideal — never mind that they can’t stop half the apps out there from going outside that paradigm and installing their crap all over the place.