SOA in Action Blog

New research from Aberdeen Group shows that to effectively test and provide QA to SOA, companies are testing not only vertically (using unit and functional testing as their benchmark for quality) but are also testing horizontally across an entire business process. In addition, in successful testing/QA environments, there is close involvement of the business user in more phases of the development lifecycle.

Aberdeen Group has just issued a report on the challenges around testing and quality assurance for SOA and Web services.

The study of 240 end-users found that so-called "best-in-class" companies are taking a multi-pronged approach to the SOA testing and QA problem by incorporating automation in the testing lab and process change at the organizational level. More importantly, best-in-class companies are increasing the involvement of business users in all phases of the development lifecycle, and view quality not as something that'’s done at the end of the cycle, but as a horizontal attribute that spans multiple business processes.

The survey found that 61% of the best-in-class companies saw a reduction in the number of defects discovered in production. By comparison, only 18% of the "laggard" companies could say they were able to reduce defects. Overall, 94% of the best-in-class companies reported an increase in the quality of deployed software, versus 17% of the laggards.

How can a company evolve into a best-in-class organization when it comes to testing and QA? The leading companies "have attacked the problem from multiple angles," Aberdeen notes. "It isn'’t enough to just deploy automation, and it isn'’t enough to simply rely on functional tests – QA for composite applications needs a horizontal, process-oriented view, not the vertical unit-test methods used in the past."

Aberdeen notes that the typical best-in-class company gets business users involved in all aspects of quality.uses automation to increase test coverage, and sees quality as more than just an end-of-the-lifecycle task.

Best-in-class companies "are almost three times as likely as industry average and laggard organizations to have redesigned their entire testing process," the report continues. "The best-in-class organizations have made the commitment to services-based applications and have retooled their key processes to reflect their new direction."

The report adds that "many of the changes are focused on bringing quality to bear on the entire lifecycle of the project by getting business users more involved. Doing this requires the top-down commitment of the enterprise. The enterprise as a whole must be willing to set aside the time for key players to be involved in projects as they move from phase to phase, and to supply them with the appropriate tools necessary."