Date Published: December 6, 2021
Comments Due:
Email Questions to:
Author(s)
Richard Kuhn (NIST), M S Raunak (NIST), Raghu Kacker (NIST)
Announcement
Combinatorial coverage measures have been defined and applied to a wide range of problems, including fault location and evaluating the adequacy of test inputs and input space models. More recently, methods applying coverage measures have been used in applications of artificial intelligence and machine learning for explainability and analyzing aspects of transfer learning. These methods have been developed using measures that depend on the inclusion or absence of t-tuples of values in inputs and test cases.
This paper introduces a new method related to combinatorial testing and measurement, combination frequency differencing (CFD), and illustrates the use of CFD in machine learning applications. This method is particularly well-suited to artificial intelligence and machine learning applications, where training data sets used in learning systems are dependent on the prevalence of various attributes of elements of class and non-class sets. This paper illustrates the use of this method by applying it to analyzing physical unclonable functions (PUFs) for bit combinations that have a disproportionately strong influence on PUF response bit values. Additionally, it is shown that combination frequency differences provide a simple but effective algorithm for classification problems.
This paper introduces a new method related to combinatorial testing and measurement, combination frequency differencing (CFD), and illustrates the use of CFD in machine learning applications. Combinatorial coverage measures have been defined and applied to a wide range of problems, including fault location and for evaluating the adequacy of test inputs and input space models. More recently, methods applying coverage measures have been used in applications of artificial intelligence and machine learning, for explainability and for analyzing aspects of transfer learning. These methods have been developed using measures that depend on the inclusion or absence of t-tuples of values in inputs, training data, and test cases. In this paper, we extend these combinatorial coverage measures to include the frequency of occurrence of combinations. Combination frequency differencing is particularly suited to AI/ML applications, where training data sets used in learning systems are dependent on the prevalence of various attributes of elements of class and non-class sets. We illustrate the use of this method by applying it to analyzing physically unclonable functions (PUFs) for bit combinations that disproportionately influences PUF response values, and in turn provides indication of the PUF potentially being more vulnerable to model-building attacks. Additionally, it is shown that combination frequency differences provide a simple but effective algorithm for classification problems.
This paper introduces a new method related to combinatorial testing and measurement, combination frequency differencing (CFD), and illustrates the use of CFD in machine learning applications. Combinatorial coverage measures have been defined and applied to a wide range of problems, including fault...
See full abstract
This paper introduces a new method related to combinatorial testing and measurement,
combination frequency differencing (CFD), and illustrates the use of CFD in machine learning applications. Combinatorial coverage measures have been defined and applied to a wide range of problems, including fault location and for evaluating the adequacy of test inputs and input space models. More recently, methods applying coverage measures have been used in applications of artificial intelligence and machine learning, for explainability and for analyzing aspects of transfer learning. These methods have been developed using measures that depend on the inclusion or absence of
t-tuples of values in inputs, training data, and test cases. In this paper, we extend these combinatorial coverage measures to include the frequency of occurrence of combinations. Combination frequency differencing is particularly suited to AI/ML applications, where training data sets used in learning systems are dependent on the prevalence of various attributes of elements of class and non-class sets. We illustrate the use of this method by applying it to analyzing physically unclonable functions (PUFs) for bit combinations that disproportionately influences PUF response values, and in turn provides indication of the PUF potentially being more vulnerable to model-building attacks. Additionally, it is shown that combination frequency differences provide a simple but effective algorithm for classification problems.
Hide full abstract
Keywords
combinatorial coverage; combination frequency difference; combinatorial testing; physical unclonable function (PUF); unclonable
Control Families
None selected
