People understandably prefer to keep control of their own safety, even if on average they are worse at it. Because individuals that do care about safety can actually do many things to lower the risks well below the statistics, and likely remain much better than self-driving.
There’s also the question of responsibility and liability. In society we have such pressures so that there’s an incentive to continually reduce risks. It is dangerous to shift the responsibility to fast-growth tech companies that will care much less about individual accidents than the drivers involved, and are more capable of absorbing liability without substantive remediation.
Safety is outsourced in many contexts (aviation, police, military…), but it has to be substantially better than what the individual can achieve on their own. For instance, due to the expertise required, or because you can achieve more through coordinated collective action. And there has to be very high trust in the companies involved, the regulatory systems around them, and the technologies employed.
I disagree that the safety concerns have to be on an individual basis, no other driving regulation works on that basis. They are based on the statistical likelihood of an overall reduction in damages or risk. You as an individual might be safe driving 10-20mph over the limit, but the limit is there to cover a variety of road conditions and road users.
The self driving system just has to fulfil those criteria, being that overall having it is safer for the average road user than not. It, also, is not the fast-growth tech companies that decide that.
Individuals are unlikely to give up their control unless the alternative is clearly substantially better. And, as you say, there are other larger interests beyond individual preferences that need to be overcome too.
Regardless, as we have seen many times with other technologies, I think that convenience will be a much stronger driver of self-driving acceptance than the safety benefit.
Wherever you draw it, there is a group of 10-100 million drivers who will be less safe (to themselves and others) in an automated vehicle.
There’s reason to believe that automated vehicles exceed bad drivers, and may improve , but it’s no guarantee that they will exceed very good or excellent drivers.
These things are set so that, on average, everyone is safer. If allowing self driving cars raises the safety of everyone on average, it makes sense.
Personally, I would be in favour of everyone having to either use a self driving car or gain an advanced driving certificate so they could drive manually. One that only 10-20% of people could pass, this would include an in depth theoretical section as well as a 2-3 hour driving test.