So it's actually entirely rational that the bar for companies to be able to ship software that makes those fatal errors without consequence other than an insurance payout should be higher (especially since when fatal error rates can only be estimated accurately over the order of millions of miles, driverless systems are more prone to systematic error or regression bugs than the equivalent sized set of human drivers, and the cost and appeal of autonomy probably means more experienced drivers get replaced first and more journeys get taken)
This not in any way refute my argument that would also be irrational to set the safety bar for autonomous vehicles as "marginally better than humans" , given that AI failure modes are distributed completely differently from human ones, a sufficiently serious edge case bug triggered only once every hundred million miles might make the autonomous system more likely to kill you than humans[1], and for that and other reasons its almost impossible to quantify whether a particular firmware update actually is safer than the average driver (takes around >10 billion miles to approach statistical significance if you're worried about fatalities rather than only weakly-correlated scrape rates, and then you've got to wonder whether the driving conditions are well matched). Especially if we're using that statistical argument not just to license the vehicles for road use but to absolve autonomous system developers of potential criminal liability for actions taken by their software, a luxury humans that wipe out pedestrians with similar driving aberrations wouldn't get.
[1]the US had 1.38 fatalities per 100 million vehicle miles in 2023, skewed significantly upwards by DUI and other egregious driving behaviour. Less than half that in other countries with different road conditions and also more in-depth driver education. Humans have a lot of car accidents, but they also drive a lot of miles.
Ideally, driverless cars will one day be better drivers than humans and this will save tens of thousands of traffic deaths per year. Holding up progress because cars will be confused in extremely rare or improbable situations will cost more lives than it saves.
Random planters in the middle of the road? Streets that narrow and then widen? Drivers start slowly creeping along, which means they are less likely to injury pedestrians.
maybe a little biological brain engineered to think it is a car with api access to the car hardware via the llm?
imagine you get into the car and in the center console you just see a floating brain in vat like fallout
The LLM will apply the high level reasoning needed to deal with longer time horizons and complex decisions, like deciding that the best way to reach the car wash 100 yards away is by walking.
You sound like an econ prof: full of it and hand waving away with hypotheticals.