I've been spending the year assessing where enterprise architecture -- and SOA in particular -- are headed as we begin to leave the first decade of the 21st century. The good news: It's clear that EA and SOA are delivering real value to most businesses today. However as we'll see, there is also considerable room for performance improvement. Trying to map out the direction of these disciplines has resulted in quite a snapshot with many interesting details and nuances that I'll share here throughout the rest of the year.
More immediately of interest we can see some trends beginning to stand out clearly right now:
1) Modern software architecture, with SOA as the top-level organizing principle, seems to have more and more trouble keeping up with the rate of change in most organizations. Not only have the timelines and schedules of IT and architecture groups never aligned well with business activities, it's now becoming an endemic problem as I've shared experiences with architects around the world recently. There's a growing sense that the business is "pulling away" and despite enterprise architecture and SOA being intentionally proactive, the default stance is one of short-term reactive response and trying to "clean up afterward", an activity which few businesses want to pay for. In short, our traditional services delivery models seem to move too slowly these days to embrace business opportunities as they occur.
2. Consumption of SOA and service-based IT is still too low. Depressingly low in some cases. The early "field of dreams" approach to services that some organizations practiced (build them and they will come) combined with the proliferation of JBOWS (just a bunch of Web services) finally morphed at last in many organizations into more mature services landscapes with some actual governance. But now we see that the subsequent complexity, rarefied skills and tools, and additional constraints ended up stamping out a lot of SOA consumption at the edge. These days the evidence is beginning to mount (presented below) that a certain style of services -- combined with mechanisms that actively drive consumption and outfitted with real business models -- could address this. There are other emerging solutions to drive consumption of SOA as well. Without consumption and uptake, SOA cannot access the "return" in ROI. Systematic removal of the obstacles to participation in SOA is the goal in this view.
3. The focus on SOA still tends to be on the [over]engineering of seams and processes instead of removing constraints on the business and increasing ready access to value. We are still not providing good enough reach to our enterprise data, IT systems, or people. Despite heavy investments in IT for 30 years and the openness and interoperability intents of SOA, the majority of our enterprise data remains submerged and inaccessible to most business users, silos are still prevalent, and there still lacks the human dimension, the latter which can represent the use of more effective architectures of participation or the involvement of more than IT in the strategic development of SOA.
While important cross-cutting concerns -- like security efforts or the imposed siloed architectures of enterprise software suites (ERP, CRM, HRM, etc) or even turf boundaries (see Conway's Law) -- are also responsible for many of these issues, at the core seems to be a project-oriented focus instead of a product-oriented focus with SOA. This is not to say that good engineering is not important, but that we are too often aiming at the wrong part of the SOA tolerance continuum.
Looking to successful models of SOA for answers
As an industry it's also apparent that we have too few examples of good SOA to point to and use as reference models. Most success stories are behind the walls of other organizations that are just as inclined to keep the secrets of success to themselves. Best practices and effective approaches therefore are not always likely to spread when they should. In fact, it's fairly easy to argue that the evolution of SOA has often moved far faster in standards bodies than by validation of effective new ideas in the field that are only then adopted and standardized when they work well.
Interestingly, this means the openness and visibility of today's modern Web seems to provide living examples of what's effective at driving forward business results with SOA. By this I mean when it addresses the key issues around improving the response to opportunities and directly enabling self-service, all while remaining highly scalable and protecting users and data. While I've been writing about Web-Oriented Architecture (WOA) for some time now, it's in truth a parallel "track" for SOA that's evolved organically in the wilds of the online world to meet many of the same challenges that we have in our organizations today.
Key to the story of WOA is that at least one critical feedback loop is present in online businesses that is much less evident in most SOA efforts today: Their business will fundamentally thrive or die based on the adoption of their services. Most new online businesses offer an API (the Web version of SOA) at the very outset these days and it's now common for the services themselves -- instead of the visual Web pages -- to be the dominant point of usage early on. Online firms offer APIs in easy to use formats and then market and evangelize them extensively precisely because there is an urgent need and these are the best ways to drive adoption. There is also more direct and immediate business value in this model since carefully measured use and consumption is the yardstick by which online firms are measured. It's also what they monetize, an entire area that's seriously underdeveloped in enterprise SOA for both internal and external customers. This is a yardstick that would change a lot of what we focus on in SOA.
Imagine how we might behave and prioritize our efforts if our success was largely based on the number of points and overall level of consumption of SOA services, internally as well as partners and customers. Unfortunately, as Joe McKendrick noted yesterday in "SOAs too big to fail", we may be increasingly isolated from that pressure as the business becomes more and more dependent on SOA. Ironically, this seems to be encouraging an organizational structure (which often is the IT department itself) that is isolated from market pressures.
How WOA can help SOA reach the next level of performance
While there are certainly many aspects of WOA that need to have an enterprise context added to them to work for traditional businesses, you can see from the visual above that a successful and capable service-based approach has emerged naturally side-by-side with classical SOA. At their roots, they both have great commonalities (HTTP, interoperability, and recombinant software), and some key differences as well. Intriguingly however, these often line up with many of the issues we're now seeing holding back the next level in business performance of SOA:
- Responding to change faster. Reducing execution chokepoints by making SOA more self-service (running your SOA like a business) by broader audiences in the organization.
- Increasing consumption. Using new service delivery models that make SOA the easiest, cheapest, and fastest way to solve problems.
- Actively enabling access to value. Opening up broad access to data and people using models such as deeply linked REST-based data webs and open supply chains. Adopt browser-based approaches to consumption such as enterprise mashups tools and user distributable widgets that project the SOA across the organization.
We certainly don't need to throw away everything that we've done with SOA to achieve these performance improvements, far from it. However, we must change much of the focus on what we do and how we do it. There are also some key additions to technical approaches that will be required such as adopting enterprise REST, making IT systems more mashup friendly, and updating existing delivery models such as portals and intranets to be more malleable, self-service, and connected to SOA data and resources.
The bottom line: Since many of the very best examples of SOA that works are the models we seeing being proven out in large scale on the Web, we must learn as much as we can from them (and the price is right, even the most compelling lessons here are free.) While enterprise SOA will never be identical to Internet-based WOA, we can borrow the best ideas from the success stories that have emerged on the Global SOA (which is what I call the service-enabled side of the Internet).
We are also starting to see evidence that a more mature melding of Web-oriented service models with the enterprise is actually starting to happen. For example, an InformationWeek survey this year showed that on the technical side of the SOA/WOA stack, SOAP adoption appears to be dropping in enterprises (54% a year ago to a projected 42% in the next 18 months) while REST is on the rise (14% to 24% over the same time frame). The recent announcement of the opening up of EMML is also showing that WOA concepts around lightweight composite applications (mashups) have moved into SOA territory.
While emerging initiatives such as JBOSS's REST-* may or may not help with this transition, as more of the foundation of WOA is available in organizations, richer and more effective outcomes on the business side of the the WOA equation are possible as well. No doubt this will take years, but this evidence and others shows that we are already beginning to incorporate the lessons of WOA into enterprise SOA.
In an upcoming post, I'll look at emerging new SOA practices that are coming from other sources as well.
Are Web-based approaches providing insight to your SOA effort? Or are you getting effective new ideas from elsewhere?