My subwoofer makes a huge difference for me inside the car without being loud enough for you to hear it. They give a nice rounded tone to the music. As with literally everything, don't hate the people that like the sound of subwoofers - hate the people that abuse them.
I am aware, that’s why I don’t have one in my flat (and I don’t own a car for that matter so none there either) but they’re still a hell of a lot of fun.