That seems to me like a somewhat odd way to put it. From where I stand, the large majority of objection to "state monopoly on violence" comes from those who otherwise express a strongly collectivist worldview.

> If we can't agree on that baseline

Part of the point about violence is it has little to do with societal agreement, to start with. It's what happens when that agreement breaks down. And in the end, it can change the agreement.