With the advent of deep-learning, open and impartial validation of complex learning and adaptive systems is becoming ever more important. For example drones and self-driving cars operate in true life-and-death situations where biases in a validation test can result in collisions with people, property, and other vehicles. In my Scientific Computing article, “Validation: Assessing the Legitimacy of Computational Results” I noted the extreme importance of open and impartial scrutiny by peers in the field:
Validation is a critical part of the scientific process and of scientific computation. While not sexy, the validation process ensures that responsible research is being performed and that legitimate results are being produced, published and promoted. An essential part of the scientific method, it seeks to evaluate the truth and justification of a scientific belief. For computation-based science, this includes open and impartial scrutiny of the computational model by peers in the field, and the creation of unbiased and reproducible tests to justify a belief by others that the model (and its instantiation in both software and hardware) actually corresponds to some truth or physical reality.
The good news is that Freedom of Information legislation has brought to light, as reported by IEEE Spectrum, that the 2012 Google self-driving car immediately passed the Nevada driving test but that, “Google chose the test route and set limits on the road and weather conditions that the vehicle could encounter, and that its engineers had to take control of the car twice during the drive.”
This is not meant to say that the Google self-driving car project is flawed, only that the validation process performed in Nevada was clearly biased. Autonomous vehicle technology is clearly going to happen in the future, but decisions based on incomplete and subjective data will only delay and harm the process while developing the technology.
For example, as a result of this disclosure do you feel that self-driving cars should be allowed around your home, children, and work? Note that regulations governing Google’s experimental self-driving cars will come into effect tomorrow (September 16, 2014) on California’s roads, that they have driven in Florida and that UK cities are now bidding for a share of a £10 million competition to host a driverless cars trial? Note the Google test passed the Nevada test, but now what is your opinion of the test relative to your perceived safety of the vehicle in proximity to people and objects important to you?
Similarly, don’t let success on your own deep-learnings project make you think the system has learned your interpretation of the data. During the 1980s a team at Sandia trained a neural network to identify a tank vs. a car. The researchers noted that the recognition rate was extremely high (e.g. successful) but the system performed abysmally in the field. It turned out that the images of tanks that were used for training were taken on a cloudy day while the images of the automobiles were taken on a clear day. The end result was a deep-learning neural network that trained itself to distinguish cloudy from clear days rather than tanks from automobiles. Kudos to the Sandia team way back when who followed through with their research to understand what was really happening! Hopefully scientists who work on drones, self-driving cars, and other autonomous objects that have the ability to destroy people, property and lives will also be impartial and willing to subject their work to unbiased review.
Leave a Reply