Load-Testing Web-Based Applications

In today's world of e-business, your customers and business partners expect fast performance from your Web sites. They assume that pages will download immediately and that online transactions will be executed efficiently and accurately. Just as in a brick-and-mortar business, where customers don't want to stand in long lines or have their credit cards charged incorrectly, they also expect business on the Internet to be conducted efficiently and without mistakes.



In order to compete effectively in the online consumer and B2B marketplaces, companies should evaluate the ability of their Web-based applications to accommodate multiple, simultaneous users viewing a Web site or engaging in online transactions. This is typically accomplished through the purchase of an application load-testing tool. Load-testing products simulate the experience of live users with "virtual" users created by load-testing technology. The load-testing software can then record the virtual users' experiences and provide this information to the client business.

Load-testing software is often distributive in nature. It is applied on multiple servers running simultaneously, with each server simulating multiple virtual users. In many cases, the testing company has developed its own proprietary browser that can be combined with a set of instructions tailored to the testing of each client business. The testing company maintains ongoing records of the virtual users' experiences at the test site, including response times and errors.

Many testing companies also monitor the client Web site remotely to help diagnose connectivity problems. The actual error messages experienced by the virtual users may be recorded for later review. A set of logs can be created that document each of the user experiences, and this information can later be compared with CPU and database testing information obtained during the test by a systems administrator or by software offered by the testing company.

One feature that load-testing companies often provide is testing the Web-based application externally from multiple points of presence to ensure that the quality of the provider's connectivity isn't the cause of system slowdowns. For example, if the client network was expected to have 10 Mbps of bandwidth but consistently experiences network slowdowns at 6 Mbps, this may signify that the network isn't getting its expected 10 Mbps.

The next aspect of the testing is to provide performance information on the infrastructure of the client network itself. Firewalls, routers and load balancers may all come into play. For instance, a firewall may not have sufficient throughput to accommodate the number of simultaneous users.

From this point forward, the testing software essentially mirrors real user activity at the site and focuses more on the performance of the application and the database software under stress.

In July 2002, Amplitude Research conducted an online survey to obtain the opinions of technology-sector employees regarding their experiences with load testing of Web-based applications. The 319 respondents represented a broad range of technology-sector professionals, from CEOs to project managers to programmers.

Factors for Successful Load Testing

Overall, the survey results indicated generally positive evaluations of the load-testing industry, the activities provided by vendors and specific vendors. Respondents who were currently conducting load tests placed great importance on the technical components of load testing while preferring an outsourced test environment. The factors they considered most important included the following (in decreasing order of importance):

  • Testing at different speeds. The reason for testing at different speeds is to determine if slower connections use more resources. This may reduce the number of virtual users who may simultaneously visit a Web site.

  • Testing on different browsers. This is not really a load-testing issue, but it can offer important insight into the error-free performance of Web-based applications on different browsers.

  • The ability to support complicated scripts to simulate user experiences. To simulate a real user experience, the load-testing company needs to script the test and provide instructions to the testing browsers. This experience needs to closely resemble the transactions performed by real users at the Web site.

Additionally, our survey results indicated that people wanted easier-to-understand reports. Customers gave the lowest comparative rating on the survey to the user-friendliness of the reports offered by current testing tools. Given the importance of this activity, vendors should invest considerable time to develop concise and easily understandable reporting methods of the testing data.

Customers who reported the highest levels of overall satisfaction with their load-testing applications were those who believed that their products did a good job of predicting real-world performance and, secondarily, provided easy-to-understand reports. Price was not a statistically significant predictor of satisfaction with a vendor, nor was the total IT budget of the respondent's company. Overall, individuals currently using load-testing tools placed greater importance on the various activities provided by vendors than did individuals who were not presently conducting load tests.

Conclusion

In evaluating a load-testing product, it is important to closely examine the load-testing software's ability to simulate the real user experience at your Web site. This requires evaluating your individual needs and verifying that the testing company can adequately meet your needs through scripting, monitoring and other relevant activities. The more information that the testing software can accumulate about your Web site and the user experiences, the more likely it is that any problem areas can be identified.

It's also important to evaluate what reporting information and advice the testing company provides after the test is completed. You should ask to see a copy of a sample report and examine what information is actually provided. Additionally, it's important to evaluate the experience of the load-testing company and whether the type of projects it has completed are similar to your requirements. The recommendations it provides after the test is completed will often be the key "takeaway" from the load test.