I'm mostly ambivalent about it excepting those situations where it makes things confusing. For example, I really enjoyed A Psalm for the Wild-Built which featured a non-binary protagonist referred to using they/them pronouns, but there were some sentences which became ambiguous as a result. That's something the editor could have caught and had corrected though and I wouldn't have had anything to complain about.

I don't know about others, but yes I'm entirely fine with people using singular they on me. My native language doesn't have an animate gender-neutral pronoun and I like how English gives me the option to actually be more gender-inclusive without an unnatural "he/she" construct.

If another cis man was suddenly uncomfortable with "they", I'd attribute that to some fragile sense of masculinity or a weird anti-woke crusade, as was the case with most such cases in my life, and find it personally annoying.

I don't want to deliberately make other people uncomfortable with my language, but it does feel exhausting to second-guess yourself, having to double-check if any innocuous phrase might be misinterpreted as an underhanded aggression, preempting a correction we know will be awkward for both sides.

I think we'd do better to consider context more than the specific language...