Knowledge Center

Estimation of Number of Errors. Mills Model. Part II

It is easy to notice that Mills method has a significant disadvantage. If we find a 100% artificial errors, it will mean that we found 100% natural error. But the less artificial errors we insert, the greater the probability that we will find them all. We insert a single artificial error, we find it, and on this basis declare that all natural errors are found! To solve this problem Mills added a second part of the model, designed to test the hypothesis about the value of N.

Assume that the program has N natural errors. We insert M artificial errors to it. We will test the program until we find all artificial errors. Suppose at this point found n natural errors. Based on these numbers, we calculate the value C:

The value C is a measure of confidence to the model. It is probability of that the model will correctly reject a false assumption.

For example, suppose there is no natural errors in the program (N = 0). Insert into the program 4 artificial errors. We will test the program until we find all the artificial errors. Suppose that's what we did not find any of the natural errors. In this case, the measure of confidence to our assumption (about absence of errors in the program) will be equal to 80% (4 / (4 +0 +1)). In order to bring it up to 90% we have to raise the number of artificial errors to 9. Following 5% of confidence in the absence of natural mistakes will cost us an additional 10 artificial errors. M will have to bring up to 19.

If we assume that the program has no more than 3 natural errors (N = 3), insert into it six artificial (M = 6), we find all the artificial and one, two or three (but no more!) natural, then the measure of confidence to the model will be 60% (6 / (6 +3 +1)).

From the formulas for calculating the confidence measures we can easily obtain the formula for calculating the number of artificial errors that are required in the program to obtain the necessary assurance of the evaluation:

Mills model is simple enough. Its weak point is the assumption of equal probability of finding software bugs. To make this assumption proved correct, the procedure of inserting of artificial errors should have some "intelligence". Another weak point is the requirement of the second part of the model to find all the artificial errors. But this may not happen for a long time, maybe never.