Downloadable Tools

We are developing tools to support combinatorial testing.  No license is required and there are no restrictions on distribution or use.  If you would like a copy, just send us an email request.  Please include the name of your company or university - this information helps us with maintaining management support for the project. All  software is provided free of charge and will remain free in the future.  NIST is an agency of the US Government.  

Software: Tutorials and guides:
  • Practical Combinatorial Testing - tutorial-style introduction to combinatorial methods for software testing (pdf)
  • ACTS User Guide - how to use the ACTS test generation tool (pdf
  • Combinatorial Coverage Measurement - explains various coverage measurements and how to use the tool for computing these (pdf)

Advanced Combinatorial Testing System (ACTS) can compute tests for 2-way through 6-way interactions.  An easy-to-use GUI is included, in addition to a command line version suitable for use in scripts or system calls from another tool.  A comparison of ACTS with similar tools shows that ACTS produces smaller test sets (with the same degree of coverage) and is faster than others.  ACTS is Java-based, so it will run on any Java platform.  

For a detailed comparison of ACTS with other combinatorial testing tools, see spreadsheet and graphs here.  

To request a copy, send email to Rick Kuhn -  

This software is provided free of charge and will remain free in the future.  NIST is an agency of the US Government.  Please Note:  Prior to Feb 09, the name "FireEye" was used for the ACTS tool. Some papers and articles on the site refer to FireEye; this is the same tool now called ACTS.

User guide for combinatorial testing tool here.

Who uses our combinatorial testing tool?  We have approximately 200 users as of July 2009, in IT, defense, finance, telecom, and many other industries.  Here is a breakdown of our user base.  

User Feedback
Users have been very positive, and are applying ACTS to a wide variety of software.

  •  "The tool is pretty easy to use, and has already reduced our planned number of tests by 20% or so." 
  •  "At present I am working with a development team to enable automated testing of a complex 
    rating algorithm under development within a property/casualty insurance system. The system
    component under test accepts several hundred independent input variables which are used
    to produce rating factors as output.  I have had some opportunity to use [NIST-ACTS] as a
    means to calculate test coverage and identify missing test cases in our current population of
    test data. I'm pleased to report that the combinatorial coverage data has proven useful to us
    in identifying gaps. Eventually, I would like to use this data to extend our test coverage as
    well -- at coverage strength 3 we're producing a volume of test cases that should be
    manageable when paired with automated test data generation. I can also foresee other uses
    which would pair automated testing tools with [NIST-ACTS] output to improve our test
    coverage for other systems."
  • "Our feedback on [NIST-ACTS] is extremely positive. The tool implement in a very efficient way 
    the algorithm of N-Wise reduction we were looking at. In fact, we would be interested
    including your technology in our tool [ ]."
  • "[NIST-ACTS] is a good application. It was easy to figure out and use. ... I would use this 
    application again."
  • ""I've fund it pretty intuitive and greatly helped to reduce the test size. I'll continue to use this tool 
    next opportunity."
  • " Not only does [NIST-ACTS] handle phenomenal combinatorial complexity without breaking a sweat, 
    it is also very easy and straightforward to use.   The user interface is very well thought
    intuitive, and delightfully uncluttered. "

Combinatorial coverage measurement:  a tool to measure combinatorial coverage of an existing test suite. This is currently an alpha release, but will eventually be open source.
To request a copy, send email to Rick Kuhn -  
User guide for coverage measurement tool here.

Access Control Policy Test (ACPT) tool allows policy authors to conveniently specify access control models (such as RBAC and Multi-Level models) and rules as well as access control properties.  From the specified models and rules, the ACPT tool automatically synthesizes deployable policies in XACML and generates combinatorial tests to verify security policy implementations.  Complete test cases are generated, consisting of test inputs and expected output for each set of inputs. ACPT uses ACTS to provide 2-way to 4-way combinatorial testing of policies.   To request a copy, send email to Vincent Hu -

.NET Configuration test file generator produces a collection of .NET config files from ACTS output. This tool can be used to provide combinatorial coverage for systems that have a large number of configuration options.
To request a copy, send email to Rick Kuhn -  

Sequence covering array generator
can produce arrays for more efficient testing of event-driven systems.  See chapter 5 of the Practical Combinatorial Testing tutorial, here, for an explanation of sequence covering arrays.  To request a copy, send email to Rick Kuhn -  

ACTS system definition
Defining variables in system under test

ACTS constraint screen
Option 1:  Constraints among variables can be added.

ACTS variable definition
Option2:  Different interaction strengths can be used for sets of variables.

Covering array
Finished covering array can be output as Excel spreadsheet, text or XML.