|
"Why is it so slow?"
"Is it doing anything?"
These are the types of painful questions you don't want to hear after your software goes live. It means first impressions are already going down in flames and there is little or no time to save them.
Sites or applications that take a long time to respond or behave in a predictable manner are not likely to get a second look, much less gain a good reputation. This is why testing application performance prior to release is an essential part of managing risk in any software project. It's also why performance testing should be considered when talking budget; managers and executives want to know what it is going to cost to build and maintain a system that supports the project goals. The licensing, additional modules, and extra hardware needed for performance testing can eat into the project budget quickly if left unchecked. However, there are ways to test the performance of your project while keeping the effort to a manageable set of basic tasks that get the job done without breaking the bank to get results.
Let's look at the market for load-testing tools. Vendors love to wow you with demos, videos, and even on-site presentations that show how quick and easy it is to use their tools to get results. I am often left scratching my head at the end and asking, "OK, but who needs to load test the desktop calculator in Windows when I will likely be dealing with an n-tier system that has a presentation layer, business layer, and backend database?" Or, "Who is going to be writing scripts for performing searches with Google?" Promises of rapid results and ease of use are sure to fade when you create your first scenario and discover the real work to be done. Accounting for unique IDs, security tokens, and data preconditions are sure to keep you busy with any tool you choose. The short version: There is no silver bullet. We know that in the applications we test we will have to deal with security, preconditions for testing, data seeding, and configuration. So, how does one avoid the pitfalls of choosing a tool while making a good choice for their project?
The majority of systems being built today are n-tier systems that expose a platform-independent web service layer used by both the client and API programmers, and there are many tools on the market that handle generating loads for web services and HTTP-based messages. If you run a .NET shop, a lot of tools are built right into the development package, Visual Studio. Some MSDN subscriptions even allow you to use an unlimited number of virtual users just for being a subscriber, saving the cost of the tools and virtual users. This is the method I have used the most, and it is my personal "go-to" package for all HTTP and .NET web service testing. For Linux-based systems there are fewer boxed solutions, but there are plenty of free tools for testing and analysis.
You shouldn't be expected to use freeware on a multimillion-dollar project, so let the budget fit the size of the project. It is going to make your organization�and you�feel a lot better if there is someone to call when things aren't working as planned. In particular, support is certainly something you might need when the tool is behaving strangely or you need to do something more unique to your testing. That is the comfort you can get with a paid product. This is not to say that there aren't effective free ways to get support. I always try to register an account on any technical forums where folks are sharing best practices and stories of their experiences with performance testing. And one should never underestimate the power of using Google to solve any and all problems. Become very skilled at asking the right questions and you are not likely to be let down often.
How you architect your load scenarios can save you money in licensing costs. If one of your scenarios is to execute something that takes ten seconds and then to wait out one hundred twenty seconds, there is a great opportunity for savings by quickening the pace of each virtual user instead of adding more users. Make one virtual user do the work of two by shortening the wait time to sixty seconds and you can cut the number of needed virtual user licenses by half. At times, you may be able to emulate the work of five real users using a single virtual user. Look for periods where your virtual users are waiting and do some simple math around your scenarios to save. |
