Another thing that could slow the introduction of self-driving vehicles is excessive regulation. Regulators have a natural tendency to be risk-averse when approving new products, because they receive far more blame for approving bad products than for delaying good ones. And especially in this case, delays matter. As we've discussed, self-driving cars have the potential to save tens of thousands of lives every year. If automakers produce a safe self-driving car, every year of delay will mean thousands of unnecessary deaths. Even worse, if automakers believe there will be regulatory delays, they may devote fewer resources to developing self-driving technologies in the first place.
Self-driving vehicles will sit at the intersection of two industries that are currently subject to very different regulatory regimes. The automobile industry is heavily regulated, while the software industry is largely unregulated at all. The most fundamental decision regulators will need to make is whether one of these existing regulatory regimes will be suitable for self-driving technologies, or whether an entirely new regulatory framework will be needed to accommodate them.
The lack of regulation in the software industry has allowed the software industry to progress rapidly. However, given that self-driving cars will have the potential to kill not only their passengers but innocent third parties, there will be significant pressures for greater regulation. On the other hand, the regulatory regime that currently governs the automotive industry is probably not a good fit for self-driving technology. It is designed for today's automotive marketplace, in which car designs evolve relatively slowly and incrementally. Software is inherently different from mechanical products like cars, and policymakers will need to approach it differently.
Concepts from the software engineering world may help both manufacturers and regulators verify vehicle safety without unduly burdening innovation. First, emulation will allow testing of self-driving software against real-world scenarios without putting real human lives at risk. Second, a layered architecture can ensure a high degree of reliability at lower layers of the software stack, while leaving space for experimentation at the higher end.
Long before the first self-driving cars are introduced on real city streets, there are likely to be traditional human-driven cars whose advanced safety features require many of the same sensors used by self-driving software. Brad Templeton tells Ars that this will create a good way for prerelease testing of self-driving software. Data can be collected from the cars of thousands of real-world drivers (with their consent of course), anonymized (we hope), and combined into a massive database of test scenarios. New self-driving software can then be run against this database to see how it would perform in a wide variety of real-world driving environments.
Hopefully, the software would make the same decisions as the human driver. Cases where the software made different decisions than the human driver could be randomly reviewed by human beings to ensure that the car's decisions were reasonable. New driving software can log millions of hours of "practice" driving on virtual streets before its first hour on a real street. This should allow even the first generation of self-driving cars to be significantly safer drivers than the average teenager.
As self-driving software matures, it will likely be possible to automate the process even further by running software known to be relatively safe alongside new, experimental software. Each software could monitor and rate the decisions of the other, and human intervention would only be required when one program made a decision that the other regarded as unsafe. Over time, a growing test suite of real-world scenarios and an emerging consensus among software programs about which decisions are safe will allow extremely thorough automated testing of new self-driving software.
The uncrashable car
The other key strategy for ensuring self-driving car safety is modularity and redundancy. The winning vehicles in the DARPA Urban challenge had many redundant components to ensure that a single hardware failure would not jeopardize the vehicle's success. Commercial vehicles are likely to follow similar principles.
Stanford's 2005 winning Grand Challenge entry
One likely approach is a layered approach similar to the design of modern operating systems. Just as modern operating systems have a simple kernel responsible for guaranteeing the integrity of the overall system and more complex applications designed to accomplish specific tasks, self-driving cars are likely to have a low-level module focused on safety and a navigational module that makes higher-level decisions.
The lower layer of software would be as simple as possible, both in its internal design and in its interaction with the higher-level software. It's likely to be physically separated and its interaction with the higher-level software carefully controlled to minimize the danger of corruption. And it should be designed so that it can intervene and safely recover any time the higher-level software tries to do something dangerous. By testing this lower layer of software thoroughly and changing it infrequently, it should be possible to guarantee a high level of safety even as more experimental navigational software is run at higher layers of the software stack.
A design of this sort seems especially likely to emerge in light of the prediction we made in the first part of this series that self-driving technologies are likely to begin their lives as advanced crash-prevention software. Automakers are likely to first develop a car that is very difficult for human beings to crash. It will then be relatively easy to replace the human driver with navigational software. Putting even buggy navigational software in such a crash-proof car should be reasonably safe, because the crash-prevention layer will rescue the navigational layer from most of its mistakes.
A layered design makes especially good sense when we consider that self-driving cars are likely to have always-on network connectivity, which creates a risk of malware infections. One hopes that self-driving cars won't run standard desktop operating systems, but gaining control of thousands of vehicles would be a juicy enough prize that someone might go to the trouble of building custom malware for a self-driving car's operating system. Therefore, a self-driving vehicle must be designed to operate safely, even if the higher layers of the software stack are compromised.