The Robot Car of Tomorrow May Just Be Programmed to Hit You. Driverless cars could be programmed, where a serious crash is unavoidable, to choose a harm reduction strategy. Maybe it will crash into a bigger rather than smaller car. Maybe it will drive you off a cliff to avoid running into a minivan full of children.
I don't doubt that driverless cars, if widely adopted, would almost certainly reduce traffic accidents and have many other benefits.
There problem is that when they DO get into accidents, it will almost certainly be because the machine engaged in behavior that a human driver would not have - driving you off a cliff to prevent you from plowing through the annual Handicapped Orphans With Puppies Parade, for example. Even if that was the "rational" decision, it will be very hard for people to accept, from both a psychological and legal standpoint.
But I hate driving in the city, and I'm terrible at it, so I say bring on the robot chauffers.