Agile SOA: Mad Science or Solid Reality?

The age of the monolithic enterprise IT structure has been over for some time -- now it's just about choosing the right nimble technology strategy. Do you attempt an SOA implementation? Do you adopt Agile methodology for internal application development? Or do you commit the unthinkable and create an unholy union of the two, giving life to "Agile SOA?"

While the development and architecture camps might seem far apart, Agile SOA isn't quite the voodoo it appears to be. Both sides have the same ultimate goal -- to enable the business to change fast. SOA provides a flexible architecture that enables IT to support business change more nimbly while Agile methods enable faster development and deployment of software. Agile methods and techniques go far beyond just building business applications. They can also evolve SOA architecture and components from the highly complex, highly risky and insanely expensive deployments they are perceived as today to something more palatable.

Agile techniques and strategies are designed to improve simplicity without sacrificing functionality or flexibility, manage risks and abate costs. But to adapt the Agile method to SOA, you need to meld the methodology to architecture -- heresy to some, but perfectly sensible to those who understand the value of Agile.

To the Laboratory!

SOA is more than an IT-only initiative. It should be driven, justified and implemented with the business and requires close collaboration, much like any joint-business venture. This collaboration between IT and the business at-large is a basic tenet of the Agile approach as well. Architects should steel themselves, however, as businesses, driven by market forces, want change now. SOA, however, requires upfront, well thought out work. This requirement for change "now" is why SOA implementation needs to unfold strategically and tactically at the same time.

Igor, Bring Me a Brain!

Strategically speaking, you need a roadmap with some amount of infrastructure in place and an idea of what applications will be deployed first. But just how much infrastructure should be implemented before you start reaping SOA benefits? What is the tipping point if SOA benefits can only be achieved by deploying applications that use it?

To reach this tipping point, you need to establish the minimum foundation for SOA services, specifically the services that implement architectural patterns not driven by the business. These are infrastructure services such as security, messaging, and interface (e.g., SOAP, WSDL, JMS, CORBA). This foundation dictates the ability to cope with business changes and refactoring applications as the enterprise moves towards SOA maturity.

But what should the minimum foundation be? Two core principles of Agile development function as a guide here, specifically leanness (build only what is needed) and incremental development (build only when needed). First, look at the initial applications to leverage the SOA infrastructure. Authentication and authorization will be required in almost every instance, so these services must be built (if not already available). But if the applications require only Web services, then keep lean and only build a Web services foundation. There is no need to build, for example, JMS or CORBA interfaces.

Similarly, the required governance should be based on the needs of the application so that these procedures and controls can be tested and proven -- building the governance incrementally as the applications require. This will save time, effort and bring about an earlier realization of benefits all around, allowing the SOA strategy to pay for itself in the long run.

It's Alive! Alive!

Now you know what foundation services need to be available, so then what applications do you deploy first? Should deployment priority be based on services provided by one application to another or on applications that result in building more of the SOA infrastructure? Or do you defer to the business need? Since agile is a collaborative approach and SOA should be driven by the needs of the business, then the answer is the business need. To help with this criterion, enter business process management.

BPM was once solely a means to define integration points between process-supporting applications; now, however, it has a more prominent role in driving SOA. While this shift is positive, it can easily be misunderstood, stemming from the fact that business processes can be modeled without regard for supporting applications or how these applications may evolve. Controls, business rules, and even user interfaces are often applied to processes to remove application dependency because the applications still can't change as fast to support the business. Unfortunately, this defeats the purpose of going SOA.

While a complete SOA implementation can provide a level of nimbleness, converging the lifecycles of process and application development can take business agility to an entirely new world. The benefits of this approach include enabling the teams (BP and AD) to keep both processes and applications in synch as well as leveraging the application to guide users through decisions and drive process adoption.

I've Created…Something Fantastic!

SOA and Agile have more characteristics in common than it looks at first glance. Both require a paradigm shift, but these shifts are complementary to one another. Both are grounded on collaboration with the business, governance in approach, flexibility, and value to the business. But one agile tenet from which SOA implementations can definitely benefit is incremental delivery. With this tenet ROI is achieved much faster, businesses see applications sooner, surprises are reduced all around, and SOA's value to the business and IT is proven.

SOA is not some mammoth, costly, overly complex monster that requires every single component to be in place for you to actually see its benefits. In actuality, it can be built incrementally and potentially funded through business projects -- the same way agile methodology functions (successfully) in business.

So in short, what does this Agile SOA creation look like?

It's about the business, not the technology. Keep your focus on the business by delivering applications early and often, and along the way support application delivery with SOA capabilities and features -- the de facto approach for most agile models.

Devise a sound architectural framework but deliver incrementally. Start with the basic service components and implement them incrementally alongside the appropriate business application. Chances are that you may already have some of these components available -- again, a common agile practice.

Other processes and services will need to be established as you evolve and as you move toward SOA maturity. It's a learning process.

At first, agile SOA might seem like a mismatched and poorly thought-out IT abomination, but as you move through the implementation, it will become abundantly clear that it's not voodoo technology but rather a tech match made in heaven.