The timing for gathering test metrics depends on the test metrics identified. For example, if metrics are gathered on the results of design inspections, the gathering occurs during the structural design stage. If metrics are gathered on defect analysis of system test results, the gathering occurs during development.
Exercise caution when gathering test metrics data. To ensure the information provided is accurate and consistent, the definitions of the information required must be clear and understood by the collector. Also, the method for collecting the information must ensure that the applicable metrics information is gathered. Effective ways to gather test metrics include:
�train collectors,
�use reliable collection mechanisms,
�provide feedback to collectors.
Train Collectors
To obtain the desired information from the collectors, you must ensure that the test metric definitions and reporting requirements are understood. One way to accomplish this is to provide a short training course for applicable personnel, to explain the information needed and how the information will be used. Review the test metric definitions and reporting requirements thoroughly, and use realistic sample exercises to reinforce and ensure understanding. Cite examples where these kinds of metrics have been useful and illustrate the benefits.
Use Reliable Collection Mechanisms
To be useful, test metrics need to be comprehensive. To obtain comprehensive information, the reporting procedure must be reliable. Whenever possible, use automated methods for gathering the information. When relying on collectors, ensure reporting is easy and a matter of routine. Ideally, the reporting should be made a part of established reporting mechanisms, (e.g., time reporting, additional information reported on a fault report). When information can not be collected through automation or existing reporting mechanisms, use other methods that prompt the collector, (e.g., forms).
Provide Feedback
Once collectors are trained and mechanisms are established for reporting the requested test metrics, it is important to monitor what is being reported. Initially, review at least five reported test metrics for each collector, then meet with the collector to substantiate the findings and provide any additional clarification, as necessary. Also, take the opportunity to thank the collectors for their time and effort.
Documenting the Test Metrics Collection Mechanism
A template and sample illustrate how to document the collection mechanism. Add this information to the requirements defined during the test metrics identification process.
