We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Business-Driven Architect

Brenda Michelson

Cloud Slam: Tom Lounibos, SOASTA, Cloud Testing: Reliability as a Service

Vote 0 Votes

Abstract: "We all saw the recent headline: "Obama Inauguration Brings BBC and CNN to a standstill".  In these hard economic times companies are increasingly relying on the Web as their main sales channel. Its use has moved from a convenient option to something endemic. Expectations for reliability (availability & speed) have become much higher. We want the latest updates and headlines, the best sources, and the most answers in the shortest time possible, and with the least amount effort and constraint. Amazon recently estimated that 100ms of latency cost them 1% of their book sales. As such, Web performance has become the new customer service.

Understanding what the user experience will be before you "go live" is more and more critical. Performance or stress testing is no longer an option, it's an imperative. In this session we will discuss the value of Cloud Testing and real world case studies of customers using it to ensure Web reliability and performance."

Tom Lounibos is the President and CEO of SOASTA, the company behind CloudTest.

Tom is talking about consequences of reliability issues.  Interesting to note recent reports where latency is now being measured in lost sales:

  • Amazon loses up to 1% of Sales with 100ms of latency on Site
  • Google loses 20% Traffic with .5 second of Latency

Calls out recent crash headlines: 40 million users crash Tata Motors site, J. Crew Website Crashes due to people searching for Obama Items, 11-hour Netflix outage

Root Causes of Poor Performance: Application - Network - Environment:

- Sudden Increase in User Activity

- Sustained Increased in User Activity

- New Application Features

- Changes in your Deployment Environment

- Changes in your Application Software

- Changes in an External Component Used

Web Applications are Built Differently: internal & external components assembled into a highly scalable web service.  Tom gives example of an application tying in Facebook, PayPal, external content distribution network and internal database.

Traditional Approach - Internal Testing

- Limited ability to test at web scale

- limited ability to simulate real world user activity

- limited ability to test external components

- limited ability to test firewalls and load balancers

- incredibly expensive: hardware-software-people

- incredibly slow process: 8 week advance notice

Cloud Testing - externally load testing your web application and networks with real world user traffic; your web application (being tested) can be behind your firewall, at a managed hosting provider, or deployed in the cloud.  The "cloud testing" part is using the cloud to produce real world load (geographic and volume) against your web application.  The testers are virtual users running actual scenarios, not just hits.

Type of Cloud Tests:

- Stress Test - where will application break, find stress point

- Load Test

- Performance

- Latency Test

- Browser Performance

Cloud Testing is delivered on pay-per-use model, pay for test hours used

Tom did an admitted mini-commercial for SOASTA, his company.  Now, he's giving customer case studies.  First customer, TurboTax to understand how the tax filing site would perform on tax day.  Spoke of issue a couple of years ago that resulted in the IRS extending tax filing deadline. 

I snapped some screen shots of the TurboTax case study slides:

Next example, Hallmark.com for the e-card superbowl, Valentine's Day.  This test plan shows both ramp-up and ramp-down.  Can cause application problems if you ramp down too quickly.  Need to see how application responds.   Another example, QTRAX, a new online music service.  QTRAX test plan (and execution) went up to 500k concurrent users.

In closing, Tom asks and answers: Why Cloud Test?  Achieve Web Scale.  Real world test.  Easy-fast-affordable. 


[Disclosure: SOASTA has done business with my company, Elemental Links, but is not a current client.]

Brenda Michelson, Principal of Elemental Links, shares her view on architectural strategies, technology trends, business, and relevance.

Brenda Michelson

Brenda Michelson is the principal of Elemental Links an advisory & consulting practice focused on business-technology capabilities that increase business visibility and responsiveness. Follow Brenda on Twitter.


BDA Feed
BDA Comments Feed

Enter your email address:

Delivered by FeedBurner

Recently Commented On

Monthly Archives