Individuals are unlikely to give up their control unless the alternative is clearly substantially better. And, as you say, there are other larger interests beyond individual preferences that need to be overcome too.
Regardless, as we have seen many times with other technologies, I think that convenience will be a much stronger driver of self-driving acceptance than the safety benefit.