March/April 2019

On Ethics
Researchers Test the Moral Dilemmas of Autonomous Vehicles

As the development of autonomous and unmanned vehicles progresses, concerns over the ethical and moral considerations of this evolving technology grows. A team of MIT researchers are addressing these questions in an experiment focused on infusing ethical principles into design and policymaking decisions.

The Moral Machine experiment began at MIT in 2016 to gain an understanding of how people make choices and how they perceive “machine intelligence” when making what could be life or death decisions.

In a paper published in the fall, the researchers outlined the results of their study, in which they used the experiment to collect decisions from 40 million participants in more than 230 nations and territories. This helped them document variations in ethical and moral preferences based on demographic indicators in addition to cross-cultural ethical variations. The results, they believe, will contribute to developing global, socially acceptable guidelines for machine and autonomous vehicle ethics.

To crowd-source opinions on how machines should make decisions when faced with moral dilemmas and consequences, the researcher launched the website http://moralmachine.mit.edu. Scenarios created by the team focused on nine factors: sparing humans (versus pets), staying on course (versus swerving), sparing passengers (versus pedestrians), sparing more lives (versus fewer lives), sparing men (versus women), sparing the young (versus the elderly), sparing pedestrians who cross legally (versus jaywalking), sparing the fit (versus the less fit), and sparing those with higher social status (versus lower social status).

In one scenario, a self-driving car will experience sudden brake failure and will crash into a concrete barrier, possibly resulting in the death of four babies and/or a cat. In another scenario the car will experience brake failure, swerve, and drive through a pedestrian crossing, possibly killing five elderly women (crossing legally).

The strongest global preferences involved sparing humans over animals, sparing more lives, and sparing young lives. The researchers assessed individual variations for participants based on their age, education levels, gender, income, and political and religious views. Even when considering all six demographic factors, they didn’t find any significant preferences that would be noteworthy for AV technology developers or policymakers. For instance, male and female participants exhibited a preference for sparing females.

The researchers found the following were the four most important cultural and economic predictors of preferences:

1. There are differences between individualistic cultures and collectivistic cultures. In individualistic cultures, there’s a strong preference for sparing more people. While in the collectivists culture, there’s a weak preference for sparing younger people due to the cultural emphasis on respect for the community’s older members.
2. On the moral questions around pedestrians who cross legally versus illegally, the results revealed that participants from prosperous (based on GDP) nations with “rule of law” institutions correlated with a preference against pedestrians who cross illegally, while participants from poorer nations and weaker institutions were more tolerant of pedestrians who crossed illegally.
3. The treatment of scenario characters differed between participants from nations with high economic and social inequality. Countries with less economic equality between the rich and poor also correlated with less equal treatment between rich and poor characters.
4. Overall, participants, despite their country of origin, showed a preference for female characters in the scenarios. This was the strongest in nations where there is less devaluation of women’s lives.

The researchers hope the experiment’s results will encourage discussions and the creation of an ethical and moral framework for policymakers as well as companies developing the technologies. They believe that all stakeholders should pay attention to the strongest preferences for sparing human lives, more lives, and young lives.

Access the paper The Moral Machine Experiment at www.nature.com.

NSPE on AV Technology and Ethics

As a leader on ethical engineering practice, NSPE believes that because the development of autonomous vehicles will have profound impacts on the public health, safety, and welfare, the testing and deployment of such technologies must include a licensed professional engineer (NSPE Position Statement No. 1772).

To promote public safety for all who interact with autonomous vehicles, NSPE has adopted a policy guide providing public policy decision-makers, regulators, manufacturers, and others with guidelines to measure the safety readiness of autonomous vehicles under consideration for deployment.

Access information on NSPE’s role in guiding safety in autonomous vehicle technology development and research.