SOA: Savior or Nemesis?

The hype around Service Oriented Architecture (SOA) is tremendous—it will revolutionize IT, make life easier, streamline business, and provide untold flexibility and responsiveness. Of course, it’s not quite that easy, especially when you already have layers of legacy technical architecture, a high maintenance and support burden, and rabid business clients demanding new solutions yesterday.

So, how do you realize the potential of SOA without raising expectations so high that they will come back to bite you? Which issues do you need to address first? And, what is a realistic timeline and approach?

Faster, Faster, Faster!

Over the last 20 years, the pace of business has quickened. Things that used to take days or weeks now take minutes or seconds. Products that used to sell for years or even decades are now obsolete in just two or three years. As a result, business is being forced to react to change far more frequently. And, that pressure to be responsive and handle rapid change is being passed directly on to IT.

Layer Cake Architecture

Unfortunately, in most large organizations, it is impossible for IT to keep up. Decades of investment in IT have left many with an architecture that resembles a layer cake. Solutions have been created by adding layers of new technology on top layers of old technology.

At the time it made sense—we were leveraging existing investments, avoiding reinvention of the wheel, and not fixing what wasn’t broken. However, the problem now is that changing a system that consists of anywhere between four and seven layers of mainframe, mini, client server, ERP, N-tier, web, portal, and integration technology is a nightmare. And, because every layer still needs to be maintained and supported ‘lights-on’ costs are huge.

The IT Catch 22

CIOs and IT departments are in a Catch 22. In order to become more responsive, more cost effective, and more flexible, IT must first become less responsive, more expensive, and less flexible. It’s the same as a highway expansion project—while you are constructing the new lanes and mass transit that will reduce traffic jams, you need to close some of the existing lanes.

For IT to deliver faster, better, cheaper solutions and be more responsive to business change, the layer cake architecture will need to be replaced. But, even if a CIO can get the organization to buy off on the potentially massive investment necessary to replace its legacy systems, what should they replace them with? After all, there’s no point in replacing legacy with solutions that will be inflexible, expensive or the legacy systems of the future.

Enter Service Oriented Architecture—IT Savior

According to the analysts, the press, the software companies, and the consultants, the savior to IT’s dilemma is SOA. Of course, a lot of what’s being said right now is hype but, done right, SOA will deliver several real benefits to both IT and the business. It can lead to a simplified architecture more able to adapt to change. It can lead to standardization, allowing greater flexibility and interoperability. It can lead to greater integration of IT solutions with each other and with business processes and users. And, as a result of all of that, it can lead to IT cost reduction, reducing both new development expense and also, importantly, maintenance and support costs.

But, SOA will only provide those benefits if it is applied consistently across business processes, applications, and infrastructures. And, it will take many years of consistent and well-managed investment to realize that promise.

Keeping Up with the Jones’s

Unlike many other hyped technology standards of the past, SOA is being adopted by all the major software vendors. The next generation of software and systems will all be based on SOA.

The major application vendors are creating what I’m calling SOERPA (Service Oriented ERP Applications). Oracle is betting on Oracle Fusion, SAP has Project Vienna and ESA, and Microsoft has committed to SOA with Dynamics and Project Green.

Equally, the major vendors are all creating technology to aid what Gartner is calling SODA (Service Oriented Development of Applications). Microsoft has Indigo, IBM has WebSphere, Oracle has Fusion, and BEA has AquaLogic.

And finally, the network vendors are rapidly embracing what they are referring to as Application Oriented Networking, network devices that act as service registry and bus. Cisco has AON, IBM has DataPower, and Intel has recently acquired Sarvega.

Auto Industry Parallels

To get an idea of where this is heading, you can look at the auto industry. Like the software industry of today, the domestic auto industry of the 1980s had consolidated to a couple of large players, with increased global competition from new Asian and European competitors, growth curves had slowed, and the drive to control and reduce costs was leading to outsourcing. So, looking at the auto industry of today may show us where the software industry is heading.

Today’s auto industry is a truly global business with the same cars and brands competing in virtually every world market. Cars are constructed on common platforms using common components. Many different vehicles, brands, and manufacturers share the same platforms and components. Ford shares platforms and components with Mazda and Volvo; GM shares platforms and components across all their brands from Saab, Opel, and Vauxhall, through Chevrolet and Pontiac, to Cadillac and GMC. And Toyota uses the same platforms and components to create products targeted at very different markets: Psion for youth, Toyota for middle America, and Lexus for affluent America.

Although components continue to be manufactured offshore by a wide range of component suppliers, the cars themselves are assembled onshore, close to the consumer, where they can be customized to their desires and needs.

A Little Crystal Ball Gazing

Over the next five to 10 years, SOA will facilitate developments in the software industry similar to those that have taken place in the auto industry.

Although services will predominantly be developed offshore, applications will be assembled onshore where they can be customized to client needs. Services will come from a variety of sources, including major software vendors, open source developers, and offshore niche vendors. If a suitable existing service is not available, new services will be home grown using custom development (SODA) technologies, Business Process Management (BPM) and/or Business Process Execution Language (BPEL) tools.

Applications and services will be deployed on both public and private open platforms. Organizations will provide private service platforms within their firewalls, probably using network devices, and will deploy services and assemble applications via those platforms. Public service platforms will be provided over the Internet and applications will be assembled and deployed using open source, home grown, and micro-charged Software as a Service (SaaS) offerings.

As integration will no longer be a barrier, assembled applications will be very specific to organizations’ needs and desires. In effect, we will be back to best-of-breed, but at the service level rather than the application level. As a result, all applications will be ‘custom’ to some degree and services will be added, removed, and replaced as business needs change—think plug and play concepts applied to applications.

The Road to the Promised Land is a Rocky One

Although the future looks bright, getting there is going to be difficult, painful, and expensive. And, for most organizations, it is also going to take far longer than they might expect.

Unfortunately, you cannot buy SOA—there is no magic bullet application or product. SOA also has higher initial upfront costs than traditional IT system development, as it requires more upfront design and architecture and an initial investment in service platform and infrastructure.

SOA will also require a paradigm shift within IT teams. A cultural change to create and use reusable services will have to be facilitated. More formal methodology and tighter management and governance will have to be adopted. Carrots and sticks will need to be created to encourage and enforce reuse, and rules and guidelines regarding service ownership, sharing, and accountability will need to be developed.

Fully realizing the benefits of SOA will therefore take years, maybe a decade. And, like the highway expansion project, the situation in IT will likely get worse before it gets better.

SOA—IT Nemesis

SOA is therefore potentially very dangerous for CIOs and IT departments. Expectations are way too high and most IT client relationships are already strained. Tackling a large architecture change like SOA with an ROI that may take a decade to realize will be difficult or impossible to justify. But giving in to the temptation to over promise will likely lead to a job search within a few years.

There are other dangers as well. Done wrong, SOA will just add another layer to the layer cake and increase maintenance and support costs. So, although wrapping legacy functionality as services will provide some increased flexibility, it will come at a cost. But SOA can’t be ignored. All the signs suggest that SOA will become the default standard within the next few years so it is critical to start building the skills and culture you will need now.

Building the Necessary Skills and Culture Now

The true art of SOA is identifying and defining services at the correct level of granularity. Service identification and definition will not just be key skills, they will be critical. The best service identification and definition starts with business processes. That means IT analysts will need to become experts at uncovering, defining, mapping and preferably improving, business processes.

Equally important is a culture that embraces reuse and application assembly. Trust, and commitment to reuse, will need to be built within the team. Effective IT governance will also be critical. SOA offers more flexibility, and also requires more loosely coupled moving parts so there are more opportunities to lose control and visibility of who is responsible for what. Effective IT governance that defines and enforces the rules and policies for managing, securing, and using services is essential.

Finally, for SOA to be truly successful, a commitment by the enterprise (i.e. senior business leadership) to operate more efficiently and effectively will be necessary. That may be the biggest hurdle of all.

Show Progress without Raising Expectations

The key will be to start making some real progress with SOA while avoiding setting expectations too high. At Hitachi Consulting, we recommend starting with a small pilot project on which you can experiment, build skills, and learn from mistakes. In parallel, we suggest holding workshops to educate your team and your business clients about the benefits and realities of SOA. These workshops build support for tackling SOA and long-term transformation within IT, but also help explain the realities of adoption and the level of commitment necessary.

As a result of the workshops, you should be able to identify a friendly business unit with whom IT can partner to develop an SOA roadmap and initial project. Normally the initial project will have something to do with customer data integration, or a self-service portal, or a single view of something. It will also have a limited scope, a small number of services, several of which will be easily reusable, and will solve a key business issue. Once the project is successfully completed it can be used as an internal marketing case study to generate support for other SOA initiatives.

Conclusions and Next Steps

SOA is real and it is really coming. Unlike hyped technologies and standards of the past, all the major software vendors are adopting SOA, so there’s no avoiding it. Managed well, SOA offers IT a tremendous opportunity to make some real positive changes and lay a foundation that will allow them to become the responsive, cost-effective IT organization their business clients are demanding.

SOA should therefore be adopted as quickly as possible, but in a realistic and disciplined way. Setting realistic expectations will be critical as will reducing risk by starting small and learning from mistakes. My advice? Get proactive and start holding some educational workshops now.

About the Author

Damian Smith is the Managing Vice President, Corporate Management Solutions with Hitachi Consulting. Over his nearly 20-year career, he has worked on both sides of the fence and has advised both CIOs and their business clients on IT related issues for over a decade.

More by Damian Smith

About Hitachi Consulting

Hitachi Consulting is the global business and IT consulting company of Hitachi Ltd. We are recognized as a leader in delivering practical business strategies and technology solutions. Visit us on the web at