> charged between banks

It generally depends on the contract the merchant has with payment provider:

- some have relatively high merchant fees to cover for interchange fees

- others (generally called IC+) have the merchant pay the IC fee plus some other (generally much smaller) fee to the payment provider

In both cases it's the merchant that ends up paying them. It's not a concidence that in Europe (where there are caps to IC fees) the fees that merchants pay are generally lower.