If you hang out in a bar with KKK memorabilia everywhere - and open the replies of any reasonably popular news story on X before complaining that's not a fair comparison - people make conclusions off your presence, even if you're personally there for the tasty beer.
It's clear this is about politics, and I'm not opposed to that, Elon is not awesome, but trying to justify it otherwise seems kind of shady.
Ignoring people of any demographic or political persuasion would be a serious strategic mistake in my opinion.