Establishing Software testing Objectives is a critical part of planning the Software testing process. Defining testing objectives is also one of the most difficult test planning activities. It is difficult because humans frequently do not have a clear idea of what they want to do until they begin to do it. This means the best laid test plans change during test process execution. This is a problem without a solution, but there are some actions testers can take which will improve test planning.
The establishment of clear testing objectives goes a long way toward offsetting future execution problems. Before the tester can do this s/he must understand what we mean by the word objective.
An objective is a testing "goal." It is a statement of what the tester wants to accomplish when implementing a specific testing activity. Each testing activity may have several objectives and there are two levels of objective specification.
A test plan should contain both high-level general objectives in the overview section, and specific low-level "provable" objectives for each particular type of testing being implemented. The latter kind being operational goals for specific testing tasks. A good set of operational objectives can intuitively explain why we are executing a particular step in the testing process.
Inputs:
System Requirements Document
Software Design Description Document
Risk Score Analysis Results
Task III.I - Identify Test Objectives
Three methods can be used to specify test objectives. The first is brainstorming. The test team uses "creative" interaction to construct a list of test objectives. This is not a free-for-all process. It should be based on analysis products such as the diagrams/text of the requirements specification. It should be performed as a walkthrough of the specifications section by section.
The second approach is to identify "key" System functions. Next, specify test objectives for each function. This procedure can also be performed as a walkthrough of the functional requirements section by section.
The third method is to identify business transactions and base objectives on them. This can also be thought of as Scenario-based as business cycles could be used to drive the process.
Output: Statement of Test Objectives - The statement of the test objectives is really a statement of the test requirements. It can be created using any word processing package or spread sheet. It can also be implemented with automated testing tools. As an example, in SQA's Manager product the test objects/requirements are input as a test requirements hierarchy and are stored in the test repository. Each branch within the requirements tree can have sub-branches, and sub-branches can also have sub-branches. SQA is only one example. Other automated testing tools will have their own type of test objectives/requirements documentation.
Task III.II. - Define Completion Criteria
A completion criterion is the standard by which a test objective is measured. Completion criteria can be either quantitative or qualitative. The important point is that the test team must some how be able to determine when a test objective has been satisfied. One or more completion criteria must be specified for each test objective.
Output: Statement of Objective Completion Criteria - The important consideration is that each requirement and how it is validated is documented. Test requirements are completely useless unless they can be satisfied. Important test metrics that should be calculated and reported are the percentage of test requirements that have be covered by test cases, and the percentage of test requirements that have been successfully validated.
The statement of objective completion criteria does not have to be a separate document. It can simply be an addendum to the statement of test objectives. For example, if using SQA's Manager product, The description field that is included for each requirement could contain a statement of the requirement's validation rule(s).
Task III.III - Prioritize Test Objectives
The test objectives should be prioritized based on the risk analysis findings. Priority should be assigned using this scale.
High - Most important tests: must be executed
Medium - Second-level objectives: should be executed only after high-priority tests
Low - Least important: should be tested last and only if there is enough time
High and Medium test objectives should be assigned more resources than Low priority objectives.
Output: Prioritized Test Objectives
Task III.IV - Operationalize Test Objectives
Manual
Test objectives should be implemented manually in the form of quality checklists, with one or more checklist items satisfying a specific objective. (Single checklist items can also satisfy more than one objective, as is the case for the date field objectives).
Objectives are internally linked to testing activities because they drive the activities. The objectives can be worded in either positive or negative fashion.
Automated
Test objectives should be translated into an appropriate form for the automated test tool being used. For example, when using SQA TeamTest Test Manager a test requirements hierarchy would be created. Automated test requirements would be stored in the tool's test repository and would be used as the basis for constructing automated test scripts.
