Search CSRC

The NIST Framework for Improving Critical Infrastructure Cybersecurity ("the Framework") released in February 2014 was published simultaneously with the companion Roadmap for Improving Critical Infrastructure Cybersecurity. The Roadmap identified Cyber Supply Chain Risk Management (Cyber SCRM) as an area for future focus. Since the release of the Framework and in support of the companion Roadmap, NIST has researched industry best practices in cyber supply chain risk management through engagement with industry leaders.  In 2014 and 2015, NIST interviewed a diverse set of organizations and...

NIST regularly conducts and awards contracts, grants, or cooperative agreements to conduct research into cybersecurity supply chain risk management (C-SCRM) and related topics. The following are relevant research activities:   Cyber Risk Analytics: A NIST and GSA-Sponsored grant from 2015-2017 examining the relationship between various risk management practices and publicly disclosed breaches. The Cyber Risk Predictive Analytics Project Cyber Risk Analytics Project Review Workshop (with video) Industry C-SCRM Best Practices: Ongoing work developing case studies exploring effective risk...

ABOUT: Cyber risk has become a topic of core strategic concern for business and government leaders worldwide and is an essential component of an enterprise risk management strategy. The Software and Supply Chain Assurance Forum (SSCA) provides a venue for government, industry, and academic participants from around the world to share their knowledge and expertise regarding software and supply chain risks, effective practices and mitigation strategies, tools and technologies, and any gaps related to the people, processes, or technologies involved. The effort is co-led by the National Institute...

***Disclaimer: Items in the following lists are provided for research purposes, and do not imply endorsement by NIST.*** U.S. Government Activities / Initiatives Related Standards / Best Practices C-SCRM Research / References Involved Standards Organizations / Associations   U.S. Government Activities / Initiatives Committee on National Security Systems Directive (CNSSD) 505 - "...provides the guidance for organizations that own, operate, or maintain [National Security Systems (NSS)] to address supply chain risk and implement and sustain SCRM capabilities". Comprehensive National...

Comments Received in Response to: Federal Register Notice (June 28, 2013) Computer Security Incident Coordination (CSIC): Providing Timely Cyber Incident Response   Date (2013) Comment Received From Aug. 14 Carbon Black, (Michael Viscuso, CEO) Aug. 14 CERT Coordination Center at the Software Engineering Institute at Carnegie Mellon University (Ryan Meeuf, CERT Coordination Center, Carnegie Mellon Univ.) Aug. 14 C.I.G.N.E.T. (Vishwas Rudramurthy) Aug. 14 Internet Identity (IID) (Chris Richardson, Senior Manager, Federal...

Access control systems are among the most critical security components. Faulty policies, misconfigurations, or flaws in software implementation can result in serious vulnerabilities. The specification of access control policies is often a challenging problem. Often a system’s privacy and security are compromised due to the misconfiguration of access control policies instead of the failure of cryptographic primitives or protocols. This problem becomes increasingly severe as software systems become more and more complex and are deployed to manage a large amount of sensitive information and...

Access control (AC) policies can be implemented based on different AC models, which are fundamentally composed by semantically independent AC rules in expressions of privilege assignments described by attributes of subjects/attributes, actions, objects/attributes, and environment variables of the protected systems. Incorrect implementations of AC policies result in faults that not only leak but also disable access of information, and faults in AC policies are difficult to detect without support of verification or automatic fault detection mechanisms. Most research on AC model or policy...

Access control mechanisms control which users or processes have access to which resources in a system. Access control policies are increasingly specified to facilitate managing and maintaining access control. However, the correct specification of access control policies is a very challenging problem. This problem becomes increasingly severe as a system becomes more and more complex, and is deployed to manage a large amount of sensitive or private information and resources. To provide high security confidence levels for the nation’s critical IT infrastructure, it is important to provide a...

This ACPT version is a beta release, which includes a concise user manual, examples, and Java code. The user documentation and software will be updated in the future. Please check the web site for update information. To download the latest ACPT version (.zip file, May, 15, 2019), please contact: Vincent Hu vhu@nist.gov for the password to unzip the zip file.   The source code is also available. The Access Control Policy Tool (ACPT) was developed by NIST's Computer Security Division in cooperation with North Carolina State University and the University of Arkansas. ACPT is provided free of...

April 27, 2010: NIST SP 800-22rev1a (dated April 2010), A Statistical Test Suite for the Validation of Random Number Generators and Pseudo Random Number Generators for Cryptographic Applications, that describes the test suite.   Download the NIST Statistical Test Suite. July 9, 2014: This update has a few minor corrections to the source code. The first change corrects the non-overlapping template test to make it correctly skip bits when a sequence matches.  The second change is to correct the π values in the overlapping template test. Software Revision History August 11, 2010:...

This information is provided for historical purposes.  Papers Statistical Testing of Random Number Generators; Proceedings of the 22nd National Information Systems Security Conference, October 1999. Presentations Empirical Statistical Testing of RNGs, 1999 RSA Data Security Conference, San Jose, CA, 1/99. Statistical Testing of RNGs, ANSI X9F1 Meeting, Institute for Defense Analyses, Alexandria, VA, 4/99. Statistical Testing of Random Number Generators, The 22nd National Information Systems Security Conference, Crystal City, VA, 10/99.

Research tools to support combinatorial testing. No license is required and there are no restrictions on distribution or use. All software is provided free of charge and will remain free in the future. NIST is an agency of the US Government, so this software is public domain. You are free to include it and redistribute it in commercial products if desired. ACTS is in Java, and will run on any platform with the latest Java release (Java free download is at java.com).  There are many users running current Windows, Mac OS, and Linux systems.  ACTS Basic version download:  Java .jar file. No...

A:  All or nearly all failures involve only 1 to 6 factors The key insight underlying combinatorial testing’s effectiveness resulted from a series of studies by NIST from 1999 to 2004. NIST research showed that most software bugs and failures are caused by one or two parameters, with progressively fewer by three or more. That is, they were only revealed when multiple conditions were true.  For example, a 2-way interaction fault could be "altitude = 0 AND volume < 2.2". So testing all 2-way combinations of parameter values could detect this problem. A method called "pairwise testing" has been...

Combinatorial testing is being applied successfully in nearly every industry, and is especially valuable for assurance of high-risk software with safety or security concerns.  Combinatorial testing is referred to as effectively exhaustive, or pseudo-exhaustive, because it can be as effective as fully exhaustive testing, while reducing test set size by 20X to more than 100X. Case studies below are from many types of applications, including aerospace, automotive, autonomous systems, cybersecurity, financial systems, video games, industrial controls, telecommunications, web applications, and...

Self-driving cars and autonomous systems of all types are notoriously difficult challenges for software assurance.  Both traditional testing and formal methods are even harder to apply for autonomous systems than in ordinary cases. The key problem is that these systems must be able to function correctly in a vast space of possible input conditions.  For example, autonomous vehicles must deal with lighting, rain, fog, pedestrians, animals, other vehicles, road markings, signs, etc.  Combinatorial methods are uniquely well suited to analysis and testing for this enormous input space, because by...

The tools distributed here are used extensively in testing for security vulnerabilities.   Survey article: Simos, D. E., Kuhn, R., Voyiatzis, A. G., & Kacker, R. (2016). Combinatorial Methods in Security Testing. IEEE Computer, 49(10), 80-83. Introduces CT-based approaches for security testing and presents our case studies and experiences so far. The success of the presented research program motivates further intensive research on the field of combinatorial security testing. In particular, security testing for the Internet of Things (IoT) is an area where these approaches may prove...

Autonomous systems are increasingly seen in safety-critical domains, such as self-driving vehicles and autonomous aircraft.  Unfortunately, methods developed for ultra-reliable software, such as avionics, depend on measures of structural coverage that do not apply to neural networks or other black-box functions often used in machine learning.   This problem is recognized and teams are seeking solutions in aviation and other fields. As one notes, "How do we determine that the data gathered to train an AI system is suitably representative of the real world?[1]"  This key question is currently...

The field of formal methods covers a broad range of mathematically-based techniques for specifying and verifying properties of software and systems.  Formal methods can be very effective for certain classes of problems, but they have gained a reputation for enormous expense.  One of the greatest opportunities for cost-effective use of these methods is the union of formal methods with testing. When a formal specification can be used in generating expected test results, the cost of developing the specification can be offset by a great reduction in the otherwise high cost of producing a test...

SEQUENCE COVERING The sequence covering array construct described below was introduced in: D.R. Kuhn, J.M. Higdon, J.F. Lawrence, R.N. Kacker and Y. Lei, "Combinatorial Methods for Event Sequence Testing",   First International Workshop on Combinatorial Testing, in Proceedings of the IEEE Fifth International Conference on Software, Testing, Verification and Validation (ICST 2012), Montreal, Quebec, Canada, April 17-21, 2012, pp. 601-609.   Preprint The notion of sequence covering has been extended to ordered combination coverage, with a theorem on necessary and sufficient conditions for...

Combinatorial methods improve security assurance in two ways: Reducing vulnerabilities - Multiple studies show that about two-thirds of security vulnerabilities result from ordinary coding errors that can be exploited (for example, lack of input validation).  By identifying errors more efficiently, combinatorial testing can reduce vulnerabilities as well.  Specialized security testing - We have been able to achieve huge improvements in fault detection for cryptographic software, hardware Trojan horse and malware, web server security, access control systems, and others.   Below are some...

One of the central problems in testing is determining what output should be expected for a given set of inputs.  This question is known as the "test oracle problem".  When a test oracle must be determined manually, testing becomes extremely expensive and time consuming.  Thus a component of any test process must be efficient solutions to the oracle problem. This section discusses a variety of approaches to solving the oracle problem and reducing test cost.  Automated Test Generation Oracle-free Testing  

Latest research: Combinatorial Frequency Differencing. NIST Cybersecurity Whitepaper.- Describes measures of the frequency of combination coverage and difference between Class and Non-class elements in machine learning classification problems.  Illustrates application of these methods for identifying weaknesses in physical unclonable function implementations.  Combinatorial Coverage Difference Measurement. NIST Cybersecurity Whitepaper.- Introduces a variety of measures that can be applied to understanding differences in combination coverage. Also see our User Manual for the coverage...

This research grew out of our 2001 paper on failures in medical device software, which found that the failures were triggered by only 1 to 4 variables interacting. Surprisingly, although "pairwise" testing had been popular for many years, no one had looked at the actual distribution of failures by number of interacting factors. We continued this work and published other papers finding that all, or nearly all, software failures involve interactions among a small number of variables, no more than 6, in thousands of failure reports. Below are some of  our research areas. If you'd like to find out...

Papers Covering Array Library Seminars & Talks & Tutorial Combinatorial Methods For Modeling & Simulation Workshop Papers DOs and DON'Ts of testing

Through quarterly meetings and email list, the Forum provides our members: a venue to exchange information, share ideas and best practices, resources, and knowledge; an ongoing opportunity to leverage the work done in other organizations to reduce possible duplication of effort; and access to a community and network of cybersecurity and privacy professionals across the U.S. federal, state, and local government and higher education organizations.  Quarterly Meetings Refer to the CSRC Events Page for upcoming Forum meetings and registration information.   Forum meetings are open to...