Automated Combinatorial Testing for Software ACTS

Combinatorial coverage - case studies

Combinatorial coverage measures are used in industry for high assurance software used in critical applications.  Industry examples include the following:

Ozcan, M. Applications of Practical Combinatorial Testing Methods at Siemens Industry Inc., Building Technologies Division.   2017 IEEE Intl Conf on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 208-215). IEEE.  
- Applied combinatorial testing to Zero Defect program for industrial control systems, using mixed-strength covering arrays, “resulting in requiring fewer tests for higher strength coverage”. 

Smith, Riley, et al., "Measuring Combinatorial Coverage at Adobe", 2019 IEEE Intl Conf on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 208-215). 
- "In this paper, we report the practical application of combinatorial testing to the data collection, compression and processing components of the Adobe analytics product. Consequently, the effectiveness of combinatorial testing for this application is measured in terms of new defects found rather than detecting known defects from previous versions. The results of the application show that combinatorial testing is an effective way to improve validation for these components of Adobe Analytics."

Ericsson S, Enoiu E. Combinatorial Modeling and Test Case Generation for Industrial Control Software using ACTS. 2018 IEEE Intl Conf Software Quality, Reliability and Security (QRS) 2018 Jul 16 (pp. 414-425)
- An analysis of expert-developed tests for Bombardier Transportation. "Our results show that not all combinations of algorithms and interaction strengths could generate a test suite within a realistic cut-off time. The results of the modeling process and the efficiency evaluation of ACTS are useful for practitioners considering to use combinatorial testing for industrial control software as well as for researchers trying to improve the use of such combinatorial testing techniques."

 

 

 

 

 

Created May 24, 2016, Updated January 25, 2021