Rouge browser extensions for example could redirect you away from the bank website (if the bank website has poor security) when you go there, so even if you use the URL from the password manager, if you don't use the autofill feature, you can still get phished. And if the autofill doesn't show, and you mindlessly copy-paste, you'd still get phished. It's really the autofill that protects you here, not the URL in the password manager.
Concretely, I think for redirect browser extension users I'd use "webRequest" permission, while for in page access you'd need a content-script for specific pages, so in practice they differ in what the extension gets access to.
Likewise I have links in the bookmarks bar on desktop.
I use these links to navigate to the main sites I use. And log in from there.
I don’t really need to think that way either.
But I agree that eliminating the possibility all-together is a nice benefit of using the browser integration, that I am missing out on by not using it.