Ideally, identifying test metrics takes place at the beginning of the project, so incorporation into the appropriate activities is easy. The test metrics you wish to collect need to be:
�quantifiable,
�easy to collect,
�simple,
�meaningful,
�non-threatening.
Quantifiable Measurements
To ensure consistent comparison of findings, the method of measurement needs to be standard, concise, and quantifiable. For example, to determine the density of defects, you need to identify what metrics provide this information and a standard of measurement. For example, the test metric to gather is the number of defects and the method of measurement is lines of code (loc), (i.e., �x� number of defects per �y� loc).
Definitions must be clear and concise. For example, the definition of defect must state what constitutes a defect and the definition of lines of code must state the number of lines of code to be used as the standard of measure, (e.g., 1000). The definitions must also provide any other information necessary to ensure consistency, (e.g., if the lines of code are commented or not commented).
Easy to Collect
The information collection process must not take too much of the collector�s time, or the information will not be collected. The amount of test metrics gathered from any one group needs to be kept at a minimum, collecting only that which is most useful. Whenever possible, automate the data collection process.
Simple Information
The information collected should be simple to gather. If it is hard for the collector to determine what to measure or report, the information is likely to be inaccurate.
Meaningful Purpose
The information gathered must have a specific purpose, (or purposes). For example, the information will be used to determine the number of defects and time used for each testing phase, in order to determine the most cost effective ways to minimize errors.
The information to collect must be understandable and viewed as relevant to the collector, or the information will not be collected. For example, to make the information in the previous example relevant, explain that the findings will highlight the testing methods that work and methods that don�t work, so that employee effort is focused on productive activities.
Non-Threatening Use
Avoid using test metrics for employee evaluation purposes. Collection of information that is perceived as a threat to the employee�s job status is frequently reported inaccurately or incompletely.
Methods for Identifying Test Metrics
Start the process of identifying test metrics by listing the problems to be solved and objectives first. Then determine the items to measure and the standards of measurement to use, to achieve the objectives.
Various methods can be used to complete the test metrics identification process, (e.g., brainstorming, use of a committee composed of representatives from management and the groups that will help with the collection process).
