Ensuring that an application will meet performance and scalability requirements when deployed requires performance testing. Without performance validation, there is great risk that the application will not provide the performance necessary to adequately support the business; this will result in customer dissatisfaction, lost business opportunity, and/or lost revenue. Most software development organizations recognize the need for performance testing, and many have invested in automated tools to help execute this testing. However, despite the commitment to testing and the investment in tools and resources, many organizations still have performance issues when their applications reach production deployment. The software doesn't scale as expected; it doesn't satisfy expected response times; and performance collapses under certain conditions or at certain stages of a transaction.
When these application performance failures occur despite testing, it is typically the result of a poorly defined or implemented performance testing practice. Setting up performance testing correctly is extremely challenging. It must accurately reflect such things as the expected population of application users, the number and frequency of tasks that will be executed, the distribution of those tasks, and the parallel activities that might compete for resources. Additionally, it requires well-defined and documented performance goals or requirements from which tests can be built. Without the right test model that reflects the full range of expected or actual usage and that has measurable performance criteria, performance testing will not be successful.
Developing an accurate usage model is key to ensuring that performance testing produces valid results and accurately predicts actual application performance. Yet, most organizations have difficulty with this task. Many do not spend enough time researching and defining the basis for the model. Others focus too narrowly on meeting specific requirements or small sets of requirements (such as Service Level Agreements [SLAs]), then find that their applications succeed in meeting these specific requirements, but suffer performance failures in all other areas of use. A good performance testing practice focuses on developing a good foundation usage model.
Linking performance testing to functional testing is another key element for success. Performance testing extends naturally from the requirements management policy and functional testing practice. Ideally, the requirements define the expected execution paths through the application (based on use cases), the expected distribution of user paths being executed simultaneously, the anticipated peak traffic levels, and any SLA performance requirements you are contracted to satisfy. Additionally, functional testing implements test cases that verify whether a single instance of each expected user path operates successfully. Performance testing should then build upon this foundation by taking the functional tests and executing them in the correct combination and frequency to simulate the usage model. To assess scalability, you verify whether the functionality operates correctly under the given load. To assess SLA compliance, you verify whether the application satisfies performance metrics (such as metrics that specify an acceptable threshold for the total time of path execution or the time of each path step execution).
An effective performance testing practice involves the definition of guidelines for using performance testing technologies effectively and then implementing and integrating those guidelines (along with supporting technologies and configurations) into your software development lifecycle to ensure that your teams apply the practice consistently and regularly. It also requires a means to monitor and measure the practice's application.
Effective performance testing must also be an iterative process. It is difficult to get the model defined exactly right the first time. You must be prepared to test, measure, evaluate, and then modify your test structure based on initial test runs in order to refine the model to correctly reflect usage patterns. And as environments change and applications are modified or extended - or as business expectations change and performance requirements are modified - performance testing needs to reflect these changes. Consequently, it is important to have a well-defined performance testing practice that is repeatable, extendable, and iterative.
