The same for skiing you don’t have eyes on the back of your head, one behind you is responsible for not crashing into you.
In car you have rear mirrors but if you have something in front of you and breaking you are not going to look in the mirrors.
The only exception I could think of would be if someone intentionally jumped in front of your car, or if you did a patently and clearly unsafe maneuver, but even then I think there'd be a sort of rebuttable presumption of fault.