that's literally what it means in theory and in practice
It may seem an obvious or insignificant point, but it is critical here. If they physically oppose fascism by following and filming ICE, I'm very much on board. If they oppose it by molotoving innocent local government buildings, I am against. If both of these actions are the concept of opposing fascism, what does it mean to be against that?
Antifa are belligerants. They undermine protests by having the maturity to die for a cause but not to live for one. One can be against that without being fascist.