Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Combinatorial Methods for Trust and Assurance

Benefits

The tools developed by NIST have been shown to reduce the time required for software verification and testing, while improving test coverage.  For example, a study of their application to industrial control software [6] showed 3X greater fault detection in 1/4 of the time used for conventional test methods, or roughly 12X improvement in efficiency.  Similar results have been shown in other studies.

Comparable improvements in semiconductor design verification could result in significant reductions in engineering time and thus cost.  In addition to the cost for FPGA and IC/ASIC verification engineers, design engineers also spend 40% - 50% of their time on verification [1], increasing engineering headcount per project.  Improving the efficiency of semiconductor functional verification will allow precious engineering hours to be spent on a larger number of projects.

In addition to cost reduction for individual designs, this improvement could improve industry access to scarce semiconductor engineering and technical talent.  It is projected that the US semiconductor industry will face a shortfall of 70,000 to 90,000 workers over the next six years [7].  Demand for IC/ASIC design engineers increased at a 2.7% CAGR from 2007 to 2022, and demand for verification engineers increased even faster, at 6.2% per year from 2007 to 2022 [1].  This increase in demand is expected to continue.   

[5] https://csrc.nist.gov/Projects/automated-combinatorial-testing-for-software/combinatorial-methods-in-testing/case-studies-and-examples
[6] Li, X., Gao, R., Wong, W.E., Yang, C. and Li, D.,  Applying combinatorial testing in industrial settings. 2016 IEEE Intl Conf on Software Quality, Reliability and Security (QRS) (pp. 53-60)
[7] https://www2.deloitte.com/us/en/pages/technology/articles/global-semiconductor-talent-shortage.html?id=us:2ps:3gl:semital23:awa:tmt:021523:chip%20supply%20shortage:b:c:kwd-1240347574454

Contacts

Rick Kuhn
kuhn@nist.gov
Address: https://www.nist.gov/people/d-richard-kuhn

Raghu Kacker
raghu.kacker@nist.gov
301-975-2109
Address: http://math.nist.gov/~RKacker/

M S Raunak
raunak@nist.gov

Topics

Security and Privacy: assurance, modeling, testing & validation

Technologies: software & firmware

Created May 24, 2016, Updated June 13, 2024