FAQ: Load testing software and application performance management

In an ideal world, software professionals find and fix performance bottlenecks long before the application gets into users' hands. But more often, performance testing happens after the fact, in response to user complaints that the software runs too slowly. "Developers are always surprised when they see how [poorly] their applications perform under load," said J.P. Jackson, a project manager for the State of Michigan who oversees the performance of thousands of applications produced by a dozen different development teams across Michigan.

Load testing software is important from a business standpoint. An app that performs fast enough helps its intended audience conduct business more efficiently. But apps that lag behind leave software users frustrated, wondering if it would be faster to complete the task by hand. This FAQ identifies problems and best practices around managing performance with load testing software. It also explores the new performance challenges emerging around applications deployed on the cloud, as well those running on mobile devices.

What is load testing software?

Load testing software evaluates how the application under testing performs as the work level approaches the limits of the application's specifications.  As the work level -- that is, the number of users and the complexity of the tasks they are carrying out -- increases, performance slows. The primary goal of load testing is to define the maximum amount of work a system can handle without performance slipping significantly. The process of the increasing the load beyond its known capacity --and seeing how that load affects the application -- is sometimes called stress testing. Load testing and stress testing both fall under the umbrella term "application performance management." Originally known as application performance monitoring, APM is about finding the root causes of poor performance and fixing issues related to user interfaces, network, databases and application code.

Application performance management software is designed to pinpoint performance problems, but its true value is not realized until developers or other IT workers take steps to rework the code causing the bottlenecks in the first place.

How can load testing software help prevent business embarrassment?

It's bad enough when applications intended for employees suffer from unanticipated slowdowns. But when performance problems plague a company's online shopping site, for example, real damage can result. Customers are likely to take their business elsewhere, putting money in competitors' pockets. And the unsatisfactory shopping experience may also lead to loss of credibility and reputation for the company behind the website.

In "Performance management tips for Cyber Monday," application performance experts and Web retailers offer examples of why companies cannot afford sluggish performance -- especially on the day widely believed to be the biggest online shopping day of the year. Among the tips they recommend: Do realistic load testing to determine the baseline of website traffic, based on history, and dig deep to gain insight into the issues load testing software uncovers. For example, software developers for online shoe retailer Zappos found that they could boost performance by rethinking -- and then rebuilding -- how caching was carried out on the page.

How can project managers get developers involved in performance testing?

Developers typically get involved with performance testing after an application has been deployed. Project manager J.P. Jackson is on a mission to change that. He wants to get developers to do basic scalability testing during the development process. If he succeeds, many of the performance problems he diagnoses on a day-to-day basis could be avoided, he said.

Testing for performance issues up front is not yet a mandated practice in Jackson's organization. He works with developers on an ad hoc basis -- typically when one of the applications they oversee runs into trouble. To educate them on the value of early performance testing, he gets developers to draw a map of the application in question, identifying each point at which the app connects with databases and legacy systems. The process is illuminating, Jackson said. "Developers discover that the application is actually accessing a system they didn't know about." These connection points are often the cause of performance problems, he said.

Michael Azoff, an analyst for UK research firm Ovum, said getting developers to conduct scalability tests early in the application lifecycle is an important best practice. "But in general, early performance testing by developers has not yet been widely adopted."

What is the best approach for cloud application performance management?

On one level, cloud application performance and on-premises application performance are managed the same way. The focus is on monitoring and managing the end-user experience, said Nari Kannan, CEO of appsparq Inc., a Louisville, Ky.-based mobile applications consulting company.  "What are the end-to-end response times the user sees when using the application?"

But choosing the cloud as the deployment platform adds new complexities to APM. As a result, cloud application performance management (CAPM) is emerging as a discipline in its own right. In "Cloud application performance management: Ten insights," Kannan explains how CAPM is complicated by the dynamic nature of the infrastructure used. For the most part, cloud applications are deployed in private cloud instances first. Then, as demand increases, public cloud instances are pressed into service.

Managing performance for applications on private cloud instances is straightforward because performance managers know which private resources they are using, Kannan said. But the process breaks down for public cloud instances, because managers have no visibility into which specific physical computing resources their public cloud provider is using. The physical resources used by the public instances could be dynamically reshuffled by the public cloud provider.

Kannan said that CAPM tools, especially those that attach a copy of a monitoring agent to every private or public instance of the application, resolve a number of these challenges. Some tools even include predictive components to alert the administrator as to where potential problems exist and prescriptive components to automate a solution.

What is the best way to manage performance of mobile applications?

The challenges of cloud application performance management pale beside those that arise around mobile apps. Software professionals have limited control over mobile application performance because it's impossible to predict the location and connectivity conditions for a mobile device at any given time. "In a QA lab, you can simulate browsers and hardware, but there is no way to accurately replicate location," said Matt Johnston, vice president for uTest, a testing services firm in Stoughton, Mass. "Even if you spend millions of dollars to test your mobile app in Bellevue, Wash., for example, all you know at the end of the day is that the app worked in Bellevue." While it's impossible to test connectivity conditions -- and thus mobile application performance -- at every location, analyzing usage statistics can help software teams place mobile app testers strategically. That approach can provide a baseline of how the mobile application is likely to perform.

Theresa Lanowitz

Managing performance for apps running on mobile devices remains more of an art than a science. But getting it right has never been more important than it is now, said voke inc. analyst Theresa Lanowitz. As a discipline, application performance management is gaining importance because it affects not just the software, but the brand of the company behind that software, she said. "Application performance needs to be a full servant in the [software] lifecycle."

How do you get developers involved in performance testing? Let us know and follow us on Twitter @SoftwareTestTT.