Covering arrays or orthogonal arrays?

A
question that testers frequently have concerns the type of test array
to be used. Orthogonal arrays (OA) are well known in statistics,
and have been applied to software testing. Should testers use
orthogonal arrays or covering arrays? What is the difference between
the two?

The
short answer to the first question is that in nearly all cases,
covering arrays are preferable for software testing. This
is a consequence of the difference in definitions. In an
orthogonal array, each combination occurs only once, but in a covering array, each combination occurs at least once.
So in a covering array, some combinations will be tested several times,
which normally doesn’t matter for software testing.

Relaxing
the constraint on number of occurrences has several positive effects.
Often, an OA matching the required combinatorial test structure
does not exist. For example suppose four test factors have two test
settings each and one has three test settings; thus the combinatorial
test structure is 2^{4}×3^{1}. An OA of strength 2 matching the test structure 2^{4}×3^{1}
does not exist (it is mathematically impossible). Also, OA based
test suites may include invalid test cases that cannot be
executed. For example, suppose in interoperability testing one
factor is operating system and another factor is browser. Since
the browser Internet Explorer (IE) does not run on the operating system
Linux, the pair {Linux, IE} is invalid. Since a test suite based on an
OA includes all pairs of combinations the same number of times, this
pair will be included in some test cases and those test cases would be
invalid. A consequence of omitting those test cases is that some valid
pairs of test setting may not be tested. The limitations of OAs led to
the use of covering arrays (CAs), which have several advantages: (1)
CAs can be constructed for any combinatorial test structure of unequal
numbers of test settings. (2) If for a combinatorial test structure an
OA exists, then a CA of the same or fewer test cases can be obtained.
(3) CAs can be constructed for any required strength (t-way) testing,
while OAs are generally limited to strength 2 and 3. (4) While
generating test suites based on CAs, invalid combinations can be
excluded.