As you say, ships are moving in and out of the EU each year - the question is, how many have "back loads" - if some percentage of the ships leave Europe empty to return to Asia for more manufactured goods, then it seems very likely that they can have the containers of unwanted clothes as part of the trip.
Some of these arguments are so silly that I'm starting to understand why the EU thinks regulations are a free lunch to improve the environment with no costs whatsoever.