I think there are some scales of magnitude to this issue. I also feel like you're a bit dishonest by omitting them.

I never care a lot about cars when commuting on foot, and I don't have a problem stopping for pedestrians when driving. Pedestrians are very predictable unless they're a small child running after a ball into the street or something similar. Those problems are solved with 30-zones and street planning in most settlements. Most of the time it's just looking left and right before crossing, and in case of busy crossings, waiting for someone to safely stop for you.

Most of the hate towards cyclists stems from them having the right of way while also behaving incredibly unpredictably. In the European city I live in, most cyclists signal less often than BMW drivers, which leads to a lot of ambiguity, which kills. As such, traffic education places immense responsibility onto drivers to basically track and predict every cyclist around their vehicle, such that they avoid potential accidents.