Like, it would be very hard to accept my best friend is a child rapist just because I discovered he had hentai that featured high school girls. It is a tenuous link at best, compounded with a demand that I ignore everything I know about him.
Could I be wrong? Sure. But you're implying it should be easy to accept with your "why is it so hard?" question.