Date Published: June 2021
Comments Due:
Email Questions to:
Author(s)
Richard Kuhn (NIST), M S Raunak (NIST), Raghu Kacker (NIST)
Announcement
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However, in many cases, structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or other black-box components. Vulnerability and fault detection in such cases will typically rely on large volumes of tests, to discover flaws that result in system failures or security weaknesses.
This publication explains combinatorial coverage difference measures that have been applied to problems that include fault identification and autonomous systems validation, and documents functions of research tools for computing these measures. The metrics and tools described are introduced as research tools; later work will be useful in determining which are of value in assurance and testing or simulation.
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However in many cases structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or other black-box components. Vulnerability and fault detection in such cases will typically rely on large volumes of tests, with the goal of discovering flaws that result in system failures or security weaknesses. This publication explains combinatorial coverage difference measures that have been applied to problems that include fault identification and autonomous systems validation, and documents functions of research tools for computing these measures. The metrics and tools described are introduced as research tools; later work will be useful in determining which are of value in assurance and testing or simulation.
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However in many cases structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or...
See full abstract
Structural coverage criteria are widely used tools in software engineering, useful for measuring aspects of test execution thoroughness. However in many cases structural coverage may not be applicable, either because source code is not available, or because processing is based on neural networks or other black-box components. Vulnerability and fault detection in such cases will typically rely on large volumes of tests, with the goal of discovering flaws that result in system failures or security weaknesses. This publication explains combinatorial coverage difference measures that have been applied to problems that include fault identification and autonomous systems validation, and documents functions of research tools for computing these measures. The metrics and tools described are introduced as research tools; later work will be useful in determining which are of value in assurance and testing or simulation.
Hide full abstract
Keywords
combinatorial coverage; combinatorial coverage difference measures; combinatorial methods; combinatorial testing; fault location; software testing; structural coverage
Control Families
None selected
