There is significant grey area wrt to "doping" too in the sense that a performance enhancing drug may express as a larger than normal amount of a naturally occurring substance. So did the person dope, or is that their natural genetics? In my scheme, WHO CARES!
Beyond that, I suppose there is the usual argument against more serious and non-natural forms of doping that it is physically detrimental to the competitors and by allowing it you are encouraging or pressuring people to essentially harm themselves.
Still, competition classes could be helpful in some of the doping grey areas.
For both the genetic disorders, they would have to be beneficial or at least not an disadvantage, for elite sport activity in order to be an issue for misclassification. For a sex-determination system, they could simply add an exception for Swyer syndrome and postpone the decision until such individual presented themselves at an Olympic competition.
Lol why does this not do it?
There are currently around 10 openly transgender women in the NCAA.
Small numbers either way.
I mean the word freak in the most loving and caring way possible, mind you.
What does fairness mean in that context?
Also, there are a significant number of these sorts of arguments in high-level sports, probably precisely because these "0.1%" cases are exactly the ones that result in exceptional ability relative to norms. It's also curious that there is such obsession about naturally occurring genetic outliers with respect to females or gender but absolute silence about naturally occurring genetic outliers among men unrelated to gender. And surprise surprise the top athletes often have such outlier genetics!
If you're drawing a distinction between natural genetic difference related to only gender and no other factors then sadly it's exactly a culture war, not a war based in science or fairness.
https://pmc.ncbi.nlm.nih.gov/articles/PMC6586948/
Also warning that article has images that may be inappropriate in a public setting. I didn't realize when I linked it.