End Users and Project Software Command have to agree and work out a series of Severity Levels. The number of these levels will vary and will reveal the relative severity of problems found during the test depending on the business /commercial return.
We have a list of 6 points from most severe to the least:
- Blocker Problem; it is impossible to continue software testing.
- Critical Problem; it is able to complete a task with some difficulty, using a workaround that is not obvious and/or is hard to use.
- Major Problem; it is able to continue testing, but live this feature will cause a massive break to business processes.
- Medium Problem; it is able to continue testing, but it causes the system to produce incorrect, incomplete, or inconsistent results, or the defect impairs the usability of the system.
- Minor Problem; software testing and live operations may progress. The defect does not cause a failure, does not impair usability, and the desired processing results are easily obtained by working around the defect.
- ‘Cosmetic’ Problem; colors; fonts; sizes. If such features are crucial to the business requirements it is possible to change the severity level.
The users and executive sponsors have to determine the responsibilities and actions for each class of problems.
Even when the severity levels and responsibility for them are agreed upon by all members, the problem of distribution to the appropriate degree of severity can be subjective and open for discussion. To avoid a protracted process determining the categories of software bugs, some examples should be agreed upon in advance to make sure that no thorough disagreement or if there are any, they will be known in advance for your organization.
It is important to agree on the criteria for Acceptance testing. Because no system can be free of defects, it must be agreed upon between the end-user and supplier and the maximum number of errors for each category must be defined. For preliminary consideration it is advisable.
In some cases, users can accept some conditions for the system. Conditions must be analyzed as possible because they can seek additional functionality that can be considered as a creep. Any fix of the developers has to be carefully tested with System Testing and Regression Testing.