Antifa is an adjective that people with no connection to one another self-apply. I'm antifa, and I imagine you are too, but it doesn't mean that we've ever met or coordinated with one another in any meaningful way.
The word "antifa" is basically meaningless altogether, since virtually every person since the end of WW2 claims to oppose fascism.
If a loose-knit ideology/movement called "Anti-Rapists" emerged that evolved into a cohort of various disconnected thugs who targeted homosexuals for violence, would being Anti-"Anti-Rapist" make you a supporter of rapists or rapist-adjacent supporter?
Obviously, in the scenario you describe, people will continue describe themselves as "anti-rapist" and everybody will understand that they mean that they are opposed to rape.
There is no "loose-knit ideology/movement" called "antifa" - there are groups like SDS and Don't Shoot PDX and a zillion others who describe themselves as "antifa", using it as an adjective. I'm aware of no person or organization who has attempted to proclaim that they are the one true antifa org.
Are some people who call themselves antifa secretly fascists? I'm sure they are. So?