The purpose of business risk analysis during software testing is to identify high-risk application components that must be tested more thoroughly, and to identify error-prone components within specific applications, which must be tested more rigorously. The results of the analysis can be used to determine the testing objectives (see Step III) during test planning.
The key to performing a successful risk analysis is to formalize the process. An informal approach to risk analysis methods leads to an ineffective analysis process.
There are four main 'risk analysis' methods which testers can employ. Each method applies a different level of formality to the risk assessment procedure.
The first is Judgment and Instinct. This is the most common approach to performing risk assessment during testing. It is an informal approach using the tester's Knowledge, and experience with past projects, to estimate the amount of testing required for the current project. This can be an effective method, but the fact that the tester's experience is not readily transferable to other testers is its primary fault. In terms of the SEI Maturity model, this risk assessment process would be somewhere between levels 1 and 2. It is a repeatable approach, but is not formally written down for others to use.
The second method is Dollar Estimation. This approach quantifies business risk using dollars as its unit of measure. It requires a great deal of precision and is difficult to use because results are based on estimates of frequency of occurrence and the loss per occurrence. Thus, the precision is not always there.
The third approach is Identifying and Weighting Risk Attributes. This approach identifies the attributes that that cause a risk to occur. The relationship of the attributes to the risk is determined by weighting. The tester uses weighted numerical scores to determine what the high risk areas are. The scores can be used to decide what application components should be tested more thoroughly than others, and total weighted scores can be used to compare one application to another.
The fourth approach is The Software Risk Assessment Package. This is an automated approach that involves purchasing an assessment package from a vendor. The advantage is the ease of use and the ability to do What-If-Analysis with risk characteristics. Automated risk assessment software packages exist which use the second and third approaches above, however, testers can create their own risk assessment questionnaires with MS Word and do the what-if-analysis with MS Excel.
Any of these approaches could be used for testing projects, however, the third approach, Identifying and Weighting Risk Factors, is the most practical and applies a reasonable level of formality. It also produces the most accurate estimate of risk. This approach identifies and weights risk factors along three project dimensions.
Risk Dimensions
The risk dimensions are identified as Project Structure, Project Size, and Experience with Technology.
With respect to project structure, the more structured a project the less the risks. Thus, software development projects that employ some type of project management / development life cycle approach should be at less risk.
Project size is directly proportional to risk. The larger the project in terms of cost, staff, time, number of functional areas involved, etc., the greater the risk.
Technology experience is inversely proportional to project risk. The more experience the development team has with the hardware, the operation system, the database, the network, and the development language the less the risk.
