In order to accommodate situations described in the first part of the article there were suggested the following criteria:
- Criterion decisions / conditions coverage
- Criterion of combinatorial conditions coverage
The criterion of coverage requires to pick up such set of tests so that each branch in the program was passed at least once, and that each simple condition was carried out in both directions at least one time. Criterion more reliable than a simple covering of branches, but it has fundamental flaw: bad test of cycles.
Combining of criteria of branch coverage and conditions coverage are also doesn’t solve all the problems caused by difficult working conditions. Errors can be associated not with the value of a simple condition, but with their combination. For example, in a fragment:
If (a = 0) or (b = 0) or (c = 0)
Then d: = a / (a + b)
Else d: = 1;
The error will be revealed only with simultaneous equality of two variables: a and b. Criterion decisions / conditions coverage does not guarantee verification of such situation.
To solve this problem it was suggested the criterion of a combinatorial conditions coverage, which requires to pick up such set of tests so that any combination of simple conditions were performed at least one time. Criterion is much more reliable than the solution / conditions coverage, but has two major drawbacks.
Firstly, it may require a very large number of tests. The number of tests required to verify the combinations n of simple terms is 2 ^ n.
Secondly, even a combinatorial coverage of conditions does not guarantee reliable test of cycles.
So which criteria to use to test your program is just your choice. But you always need to find a compromise between the completeness of software testing and the number of tests.