Self driving cars will have to decide who should live and who should die.

Johnson points to a collaboration by Awad et al. that explored the moral dilemmas faced by autonomous vehicles. They designed an experimental platform (The Moral Machine Website) that gathered 40 million decisions in ten languages from millions of people in 233 countries and territories. A few clips from Johnson's summary:

The study...identified a few preferences that were strongest: People opt to save people over pets, to spare the many over the few and to save children and pregnant women over older people. But it also found other preferences for sparing women over men, athletes over obese people and higher status people, such as executives, instead of homeless people or criminals. There were also cultural differences in the degree, for example, that people would prefer to save younger people over the elderly in a cluster of mostly Asian countries.

Outside researchers said the results were interesting, but cautioned that the results could be overinterpreted. In a randomized survey, researchers try to ensure that a sample is unbiased and representative of the overall population, but in this case the voluntary study was taken by a population that was predominantly younger men. The scenarios are also distilled, extreme and far more black-and-white than the ones that are abundant in the real world, where probabilities and uncertainty are the norm.

“The big worry that I have is that people reading this are going to think that this study is telling us how to implement a decision process for a self-driving car,” said Benjamin Kuipers, a computer scientist at University of Michigan, who was not involved in the work.

“Building these cars, the process is not really about saying, ‘If I’m faced with this dilemma, who am I going to kill.’ It’s saying, 'If we can imagine a situation where this dilemma could occur, what prior decision should I have made to avoid this?” Kuipers said.

And here is a TEDxCambridge talk by Iyad Rahwan "The Social Dilemma Of Driverless Cars"