Published: August 2, 2021
Citation: Computer (IEEE Computer) vol. 54, no. 8, (August 2021) pp. 77-85
Author(s)
Jack Toohey (Loyola University Maryland), M S Raunak (NIST), David Binkley (Loyola University Maryland)
A Deep Neural Network (DNN) based system, such as the one used for autonomous vehicle operations, is a “black box” of complex interactions resulting in a classification or prediction. An important question for any such system is how to increase the reliability of, and consequently the trust in, the underlying model. To this end, researchers have largely resorted to adapting existing testing techniques. For example, similar to statement or branch coverage in traditional software testing, neuron coverage has been hypothesized as an effective metric for assessing a test suite’s strength toward uncovering failures and anomalies in the DNN. We investigate the use of realistic transformations to create new images for testing a trained autonomous vehicle DNN, and its impact on neuron coverage as well as the model output.
A Deep Neural Network (DNN) based system, such as the one used for autonomous vehicle operations, is a “black box” of complex interactions resulting in a classification or prediction. An important question for any such system is how to increase the reliability of, and consequently the trust in, the...
See full abstract
A Deep Neural Network (DNN) based system, such as the one used for autonomous vehicle operations, is a “black box” of complex interactions resulting in a classification or prediction. An important question for any such system is how to increase the reliability of, and consequently the trust in, the underlying model. To this end, researchers have largely resorted to adapting existing testing techniques. For example, similar to statement or branch coverage in traditional software testing, neuron coverage has been hypothesized as an effective metric for assessing a test suite’s strength toward uncovering failures and anomalies in the DNN. We investigate the use of realistic transformations to create new images for testing a trained autonomous vehicle DNN, and its impact on neuron coverage as well as the model output.
Hide full abstract
Keywords
autonomous vehicles, neural network, metamorphic testing
Control Families
None selected