The problem with believing all violence is illegitimate (even that which has been democratically granted to the state to enforce laws), is that society breaks down and loses its legitimacy if you remove this enforcement aspect.
The alternative to a monopoly on violence centralized in a democratic government is not zero violence. The alternative is decentralized violence (anarchy). I think everyone on both sides would find this far less desirable.