Because of that observation, I think it would be wise for you to dig deep on whether something about your own perceptions is failing you in this regard. I think there’s a strong likelihood that either something about your physical appearance looks very different, your own behavior is causing staring, or the staring isn’t happening at all in the way you describe.
Unless you just live in a specific place/culture where staring is prevalent.
I don't stare at people. They don't stare at me either. Problem solved?