Ethics are built into the way humans behave. As cars become even more autonomous, will machines be able to think the same way?
U.S. President Barack Obama addressed the issue when he guest-edited the latest issue of Wired magazine. "There are gonna be a bunch of choices that you have to make, the classic problem being: If the car is driving, you can swerve to avoid hitting a pedestrian, but then you might hit a wall and kill yourself. It’s a moral decision, and who’s setting up those rules?"
The idea that an algorithm can be used to work out the best answer to a moral question is age-old: It was captured by 17th century German philosopher and mathematician Gottfried Leibniz's Latin motto calculemus! (Let us calculate!) and in British philosopher Jeremy Bentham's utilitarian approach that the highest quantity of happiness is something to be calculated.
Now, the new generation of cars will have to grapple with the so-called trolley problem — a thought experiment in ethics in which a runaway trolley is racing down the track toward several people, and you control a switch that can send the car down a different track killing fewer people.
For cars, the question is even trickier.
Who dies and who lives?
“The easy comparison is 'Does the car choose to hit the grandmother or the baby?'” said Bill Ford, executive chairman of Ford Motor Company, in a recent speech on the subject. “But then it goes beyond that. What if the best outcome of society is to take you out — the occupant?”
"If you know you can save at least one person, at least save that one. Save the one in the car" — Christoph von Hugo, Mercedes manager of driverless car safety
Policymakers are already grappling with problems posed by cars with automatic pilot functions, like the Tesla Model S, which stopped short of removing the steering wheel but introduced divided responsibilities between car and driver.
Obama suggested that in the early stages of a developing technology, governments should have a "light touch."
"As technologies emerge and mature, then figuring out how they get incorporated into existing regulatory structures becomes a tougher problem, and the government needs to be involved a little bit more," he said.
That's happening around the world as regulators begin to grapple with the implications of driverless cars.
Germany’s Transport Minister Alexander Dobrindt has proposed that a car should never distinguish between humans based on categories such as race or age and that it should always opt for property damage over personal injury. If a driver allows the automatic pilot to take control, then responsibility for an accident shifts to the car’s manufacturer.
Not all German carmakers agree. Future self-driving Mercedes will be programmed to prioritize the car's occupants, Christoph von Hugo, the company's manager of driverless car safety, told Car and Driver magazine.
"If you know you can save at least one person, at least save that one. Save the one in the car," von Hugo said.
That brings us back to the issue of ethics. It's clear that the algorithm should opt for the less damaging solution, but sometimes, as with the trolley problem, it's not always obvious what that is. Should the car owner be able to decide how to set the algorithm according to his or her own ethical choices?
Ethics and morals are intrinsically fuzzy — no amount of rapid calculations can give them the clarity of mathematics. And if drivers, or the manufacturer (and the engineers and ethicists employed by them), are able to program the car's ethics, this changes the degree of intention of any resulting deaths, and also its legal status.
"We mean for that to happen. This premeditation is the difference between manslaughter and murder, a much more serious offense,” wrote Patrick Lin, director of the Ethics and Emerging Sciences Group at California Polytechnic State University.
“Ethical dilemmas are likely not resolvable in a way that everybody agrees with. That's why they're dilemmas, exactly because there's no clear consensus and that there's good reasons that support different answers,” he told POLITICO in email exchange.
In cultures that revere the elderly, the car would aim for the child, while in the West, which prizes youth, the car would hit the grandmother.
A car buyer of the future may have to decide the ethical makeup of the vehicle, and in some way share the resulting responsibility for the choices that car makes in the event of an accident.
In his speech, Ford called for all driverless cars to be installed with standardized ethical technology, but that doesn't solve the problem of just what morals should be imbued in the machine.
Whose ethics rule?
Should the moral algorithm built into connected and self-driving cars be based on Western philosophical tradition, or maybe Eastern, or yet another? If so, then Ford's conundrum might have a different answer. In cultures that revere the elderly, the car would aim for the child, while in the West, which prizes youth, the car would hit the grandmother.
On a higher philosophical level the implications are even more important, especially for the Western way of thinking, which prizes the individual and freedom of will.
What's new with driverless cars, when the algorithm is fully in charge, is that “the individual becomes irrelevant, that the community and especially the calculation prevail,” said Paolo Bottazzini, a former lecturer at Milan's Politecnico University and author of several books on algorithms.
“I am not sure we Europeans are fully aware that with driverless cars what's at stake is our way of thinking,” he said.
Douglas Hofstadter, a professor of cognitive science and Pulitzer prize winner, was far more blunt. He told us he finds the idea of driverless cars “repugnant, and I absolutely cannot understand why so many people think it’s great.”
This article is part of a POLITICO Special Report: The future of driving.