Right, but at least historically, what alternative is there? You can’t really unroll thousands of years of human history and make everyone go back to where their ancestors came from (even just because people ended up mixing after colonizing other places), so you have to take some state as the correct one and then condemn every change after that (or just let everyone do whatever they want).
Otherwise, how would you decide who gets which part of the world?