Actually it's the opposite. Average people speaking out is how the world gets better. It's when they don't speak out that things are allowed to get worse.
You literally live in a Democracy. There's 5.8 billion people on this planet who wish they had the kind of power you have. If you give up your rights without a fight, you don't deserve them.