Search CSRC

Fundamental background papers: Empirical justification for combinatorial testing:  D.R. Kuhn, D.R. Wallace, A.M. Gallo, Jr., Software Fault Interactions and Implications for Software Testing, IEEE Transactions on Software Engineering, vol. 30, no. 6, June 2004, pp. 418-421.Abstract; DOI: 10.1109/TSE.2004.24  Preprint.  Comment: Investigates interaction level required to trigger faults in a large distributed database system. IPOG algorithm used in construction of covering arrays:  Y.Lei, R. Kacker, D.R. Kuhn, V. Okun and J. Lawrence, IPOG: a General Strategy for T-way Software Testing, 14th...

Although most combinatorial testing problems have varying numbers of values per variable, in some cases all variables have the same number of values and a pre-computed array can be found.    NIST library of pre-computed covering arrays   Arrays are available for t=2 to t=5, with 2 to 6 values per variable, and for t=6 with 2 to 5 values per variable.    Large collection of covering arrays available for download  (Jose Torres-Jimenez)   Data on the smallest uniform covering array sizes  for up to 20,000 variables for t=2, and up to 10,000 for t=3 through t=6. (Note that this database...

Quick introductions to Combinatorial Testing: Practical Applications of Combinatorial Testing, East Carolina University, March 22, 2012. Combinatorial Testing and Design of Experiments, TU Berlin, June 28, 2011. Combinatorial Testing, Institute for Defense Analyses, April 6, 2011. (approx. 2 hours) Combinatorial Testing Seminar, US Army Test & Evaluation Command, Aberdeen Proving Ground, May 17, 2010. (approx. 3 hours). Combinatorial Testing, Carnegie-Mellon University Jan 26, 2010. (approx. 60 min.) Combinatorial Testing Tutorial, National Defense Industrial Association, Reston, VA,...

D.R. Kuhn, R. Kacker and Y.Lei, Random vs. Combinatorial Methods for Discrete Event Simulation of a Grid Computer Network, MODSIM World 2009, Virginia Beach, Virginia, October 14-16, 2009. In Selected Papers Presented at MODSIM World 2009 Conference and Expo, edited by T.E. Pinelli, NASA/CP-2010-216205, National Aeronautics and Space Administration, pp. 83-88. R. Kessel and R. Kacker, A Test of Linearity Using Covering Arrays for Evaluating Uncertainty in Measurement, Advanced Mathematical and Computational Tools in Metrology and Testing (AMCTM VIII), Paris, France, June 23-25, 2008, Series...

In 2012, we co-founded the International Workshop on Combinatorial Testing, focused on theory and application of CT.  Papers from previous workshops are listed below.    IWCT 2025 A Combinatorial Approach to Reduce Machine Learning Dataset Size Megan Olsen, M S Raunak, Rick Kuhn, Fenrir Badorf, Hans van Lierop, Francis Durso A Search-Based Benchmark Generator for Constrained Combinatorial Testing Models Paolo Arcaini, Andrea Bombarda, Angelo Gargantini Combinatorial Methods for Enhancing the Resilience of Production Facilities Klaus Kieseberg, Konstantin Gerner, Bernhard Garn,...

DON'T assume that 2-way combinations (pairwise testing) will be enough. Empirical data, documented in papers on this site, show that 2-way combinations are important, but a large proportion of faults involve more than two parameters.  but DO consider the appropriate level of t-way combinations to be used.   It is reasonable to expect that 30% or more of the faults that need to be found in testing may require three factors for detection.   DON’T try to develop the input model (the parameters and test values) only from use cases.  Considering only use cases is likely to lead to missing some...

NIST includes here a list of events which may be of interest to those involved with post-quantum cryptography.  In particular, this list is intended to include events which will promote research in the main areas involved with our post-quantum cryptography standardization project.  For example, workshops devoted to the families comprising the Round 2 candidates (lattices, codes, isogenies, multivariate, etc).  It should be noted that NIST is not affiliated with or involved with the organizing of these workshops, and is providing this list as a source of information for the community.  Any...

Authority:  This work is being initiated pursuant to NIST’s responsibilities under the Federal Information Security Management Act (FISMA) of 2002, Public Law 107–347. The submission deadline of November 30, 2017 has passed.  Please see the Round 1 Submission page for a list of complete and proper submsisions. The Call for Proposals is available for historical reference.   Background In recent years, there has been a substantial amount of research on quantum computers – machines that exploit quantum mechanical phenomena to solve mathematical problems that are difficult or intractable for...

Post-quantum candidate algorithm nominations are due November 30, 2017. Call for Proposals   Submission packages must be received by NIST by November 30, 2017. Submission packages received before September 30, 2017 will be reviewed for completeness by NIST; the submitters will be notified of any deficiencies by October 31, 2017, allowing time for deficient packages to be amended by the submission deadline. No amendments to packages will be permitted after the submission deadline, except at specified times during the evaluation phase (see Section 5). Due to the specific requirements of the...

Post-quantum candidate algorithm nominations are due November 30, 2017. Call for Proposals Those submission packages that are deemed by NIST to be “complete” will be evaluated for the inclusion of a “proper” post-quantum public-key cryptosystem. To be considered as a “proper” post-quantum public-key cryptosystem (and continue further in the standardization process), the scheme shall meet the following minimum acceptability requirements: The algorithms shall be publicly disclosed and made available for public review and the evaluation process, and for standardization if selected, freely...

Post-quantum candidate algorithm nominations are due November 30, 2017. Call for Proposals NIST will form an internal selection panel composed of NIST employees to analyze the submitted algorithms; the evaluation process will be discussed in Section 5. All of NIST’s analysis results will be made publicly available. Although NIST will be performing its own analyses of the submitted algorithms, NIST strongly encourages public evaluation and publication of the results. NIST will take into account its own analysis, as well as the public comments that are received in response to the posting of...

Post-quantum candidate algorithm nominations are due November 30, 2017. Call for Proposals NIST will form an internal selection panel composed of NIST employees for the technical evaluations of the submitted algorithms. This panel will analyze the submitted algorithms and review public comments that are received in response to the posting of the “complete and proper” submissions. The panel will also take into account all presentations, discussions and technical papers presented at the PQC standardization conferences, as well as other pertinent papers and presentations made at other...

As of Oct 24, 2024:  This information was out-of-date and there are currently no updates. All information was moved to the PQC Archive page.  

The National Institute of Standards and Technology (NIST) is requesting comments on a new process to solicit, evaluate, and standardize one or more quantum-resistant public-key cryptographic algorithms. Currently, public-key cryptographic algorithms are specified in FIPS 186–4, Digital Signature Standard, as well as special publications SP 800-56A Revision 2, Recommendation for Pair-Wise Key Establishment Schemes Using Discrete Logarithm Cryptography and SP 800-56B Revision 1, Recommendation for Pair-Wises Key-Establishment Schemes Using Integer Factorization Cryptography. However, these...

AES Overview | NIST Reports | Federal Register Notices | Rijndael Info | Related Publications AES Overview Beginning in 1997, NIST worked with industry and the cryptographic community to develop an Advanced Encryption Standard (AES). The overall goal was to develop a Federal Information Processing Standard (FIPS) specifying an encryption algorithm capable of protecting sensitive government information well into the 21st century. The algorithm was expected to be used by the U.S. Government and, on a voluntary basis, by the private sector. On January 2, 1997, NIST announced the initiation of...

Since announcing KECCAK as the winning algorithm of the SHA-3 Cryptographic Hash Algorithm Competition on October 2, 2012, NIST has consulted with the Keccak design team and the cryptographic community in its effort to specify Keccak as the new SHA-3 Standard. The table below shows major events in the development of FIPS 202, SHA-3 Standard:  Permutation-Based Hash and Extendable-Output Functions. Date Event 10/02/2012 SHA-3 competition ended; KECCAK announced as the winner. (NIST News Release) 02/06/2013 KECCAK team’s visit and presentation on SHA-3....

At A Glance   Purpose: Carry out essential activities to help prepare all levels of the organization to manage its security and privacy risks using the RMF   Outcomes:  key risk management roles identified organizational risk management strategy established, risk tolerance determined organization-wide risk assessment organization-wide strategy for continuous monitoring developed and implemented common controls identified   Resources for Implementers RMF Online Introductory Course RMF Quick Start Guide (QSG): Prepare Step FAQs Privacy Risk Assessment Methodology (PRAM)...

At A Glance     Purpose: Inform organizational risk management processes and tasks by determining the adverse impact  with respect to the loss of confidentiality, integrity, and availability of systems and the information processed, stored, and transmitted by those systems   Outcomes:  system characteristics documented security categorization of the system and information completed categorization decision reviewed/approved by authorizing official   Resources for Implementers RMF Quick Start Guide (QSG): Categorize Step FAQs Controlled Unclassified Information (CUI)...

At A Glance   Purpose: Select, tailor, and document the controls necessary to protect the system and organization commensurate with risk   Outcomes:  control baselines selected and tailored controls designated as system-specific, hybrid, or common controls allocated to specific system components system-level continuous monitoring strategy developed security and privacy plans that reflect the control selection, designation, and allocation are reviewed and approved   Resources for Implementers SP 800-53 and SP 800-53B Introductory Online Courses RMF Quick Start Guide...

At A Glance   Purpose: Maintain ongoing situational awareness about the security and privacy posture of the system and organization to support risk management decisions   Outcomes:  system and environment of operation monitored in accordance with continuous monitoring strategy ongoing assessments of control effectiveness conducted in accordance with continuous monitoring strategy output of continuous monitoring activities analyzed and responded to process in place to report security and privacy posture to management ongoing authorizations conducted using results of continuous...

A total of fifteen statistical tests were developed, implemented and evaluated. The following describes each of the tests. Frequency (Monobits) Test Description: The focus of the test is the proportion of zeroes and ones for the entire sequence. The purpose of this test is to determine whether that number of ones and zeros in a sequence are approximately the same as would be expected for a truly random sequence. The test assesses the closeness of the fraction of ones to ½, that is, the number of ones and zeroes in a sequence should be about the same. Test For Frequency Within A...

November 21, 2014: NIST requests comments on the latest revision of NIST SP 800-90A, Recommendation for Random Number Generation Using Deterministic Random Bit Generators, which is dated November 2014. This document specifies Deterministic Random Bit Generators based on approved hash functions (as specified in FIPS 180-4), HMAC (as specified in FIPS 198-1) and block ciphers (as specified in FIPS 197 for AES, and SP 800-67 for TDEA). This revision removes the previously approved Dual_EC_DRBG that was based on the use of elliptic curves and includes a number of other changes that are listed in...

SP 800-38A: Five Confidentiality Modes In Special Publication 800-38A, five confidentiality modes are specified for use with any approved block cipher, such as the AES algorithm. The modes in SP 800-38A are updated versions of the ECB, CBC, CFB, and OFB modes that are specified in FIPS Pub. 81; in addition, SP 800-38A specifies the CTR mode. In the Addendum to SP 800-38A, NIST has specified three variants for extending the domain of the CBC mode using "ciphertext stealing." SP 800-38B: An Authentication Mode The CMAC authentication mode is specified in Special Publication...

Submissions should specify a mode of operation for a symmetric (secret) key block cipher algorithm.  At a minimum, the mode should support underlying block ciphers with key-block combinations of 128-128, 192-128, and 256-128 bits.  However, the specification should be generic – i.e., written to handle other key-block combinations, if they can be supported.  Example modes include, but are not limited to, techniques for performing encryption, message authentication, hashing, and random bit generation.  It will be helpful to receive variations of Counter mode arising from alternative...

Proposed Modes This page contains links to the proposals for block cipher modes of operation (modes, for short) that have been submitted to NIST for consideration. NIST maintains this page in order to facilitate public review of the modes; comments may be submitted to [email protected]. Appearance of a mode in this list does not constitute endorsement or approval by NIST. See the Current Modes page for descriptions of the modes that are currently approved. For each proposal below, links are given to the available documentation, as described in the following list of abbreviations:...