Keys to Successful Governance with SOA

Introduction



Why have so many hybrid vehicles been registered in the state of California in the last 12 months? Is it the more than $1500 in federal tax credits given to hybrid owners, or the luxury of cruising down the high-occupancy vehicle lanes solo during commute hours? Or is it that Californians are becoming more environmentally aware? Whatever the true reason, the reality is that policies were put into place to encourage desirable behavior—the purchase of low-consumption vehicles. This is an example of governance: when policies are put into place to encourage desirable outcomes.

Some people use SOA governance to mean service lifecycle governance—that is, governing the lifecycle of services from creation through deployment. Others take it to mean applying runtime policies to services. But is there more to SOA governance than this? Shouldn’t governance with SOA ultimately be about delivering on your business and SOA objectives? Finally, without a common understanding of what governance means, are organizations that adopt SOA simply setting themselves up for failure?

This article outlines a framework for governance as it specifically relates to SOA, and introduces our Six Steps to Successful SOA Governance model. Armed with this model, architects and IT managers with SOA responsibility will have the knowledge and framework they need to help ensure SOA success.

Context

Peter Weill of MIT defines IT governance as "specifying the decision rights and accountability framework to encourage desirable behavior in the use of IT.” In other words, IT managers must use decisions, processes, and policies to encourage the behavior that contributes to success—in this case, successful SOA adoption (see Figure 1). This definition of governance implies that you have to have some concrete expectations up front. In order to deliver on these expectations, and as part of your SOA strategy, you need a plan that we refer to as the SOA Roadmap. The SOA Roadmap is simply an outline of the capabilities and policies that need to be put into place over a period of time (such as 2–5 years) to help ensure that you deliver on your business and SOA strategy. By incrementally building the required capabilities over a period of time, you can increase your SOA maturity, thereby enabling you to deliver more projects in a more efficient and change-resilient way.

Figure 1: Essence of Governance with SOA

To help ensure success, you should enact policies and supporting processes that support the delivery of the SOA Road Map. You should communicate them widely, and then monitor their implementation and make adjustments as you go. This is the essence of governance with SOA—enacting policies and procedures to ensure the timely and appropriate execution of your SOA Roadmap.

So if governance with SOA is about decisions, processes, and policies, the natural question that arises is, “What kinds of policies do you need to put into place? And to what do those policies need to be applied?”

Key Leverage Points for SOA Governance

The key aspects that need to be governed are shown in Figure 2:

Figure 2: Key Leverage Points for Policies

The key thing to understand is that you can only achieve the change necessary for SOA success by putting policies and processes in place around all of the key leverage points denoted above—people, application portfolio, services portfolio, projects, services, enterprise architecture, enterprise technology platforms, and operations. If you put these policies in place—that is, if you govern your SOA journey wisely— you will be able to deliver on your SOA strategy and business objectives. We now elaborate on the kind of policies that need to be enacted with respect to each of the leverage points mentioned above.

Architecture. Every system must be built so that it both fits into your existing environment and reflects your organization’s future vision and SOA strategy. Architectural policies provide the foundation and framework for your SOA and enable you to build such a system better, faster, and cheaper. Building out your SOA to enable change is best done using an architectural approach that sets up a minimal set of constraints, thereby realizing consistency in service implementation and improved interoperability; realizing stakeholder innovation; and enabling applications that are minimally developed, have no adverse effects on other systems, offer general-purpose capabilities that are useful to other applications, and take advantage of and enhance a shared infrastructure. As part of your SOA journey, you should consider policies built around 1) standards compliance—for example, WS-I Basic Profile compliance for service interfaces; 2) use of architectural assessments, including reviews and change processes; 3) utilization of architecture documents and guidelines covering use cases, views, service interface design, and design patterns; 4) use of service-based application blueprints; and 5) adherence to reference architectures.

There are a few things to keep in mind when planning your architecture policies. First, by keeping your enterprise architecture artifacts simple, you increase the chances that your audience will understand them, project teams will actually read them, and you will be able to effectively enforce and update them over time. Second, timely guidance from an enterprise architect who understands the current environment and the future vision for the enterprise—even when this guidance is imperfect and based on incomplete information—is often far better than the uneducated guesses that a project team will make on its own while it waits for the official architecture to be published. Organizations must attain a certain level of process maturity—and architects must establish credibility—before they can incorporate effective SOA architectural governance. Don't expect to establish governance authority before the organization as a whole sees the value of the SOA program and is ready to change its behavior to attain that value.

Technology Infrastructure. Technology must be identified, sourced, and managed just like any other component of SOA; it is not a one-time “fire-and-forget” decision. Hence, policies need to be enacted to ensure that 1) a technology foundation (often termed Strategic SOA Platform) that provides messaging, security, and other services is centrally funded and leveraged by all projects; 2) consensus is built regarding the migration of legacy systems and platforms to SOA technologies; 3) SOA platform enhancements coincide with the project portfolio plan and business service portfolio plan; and 4) the design and implementation of shared foundation services are a part of SOA infrastructure. If each of your project teams uses its own separate infrastructure for SOA—without integrating or sharing components with the other teams—is there any chance of building composite applications that are reliable and secure? Many architecturally mature organizations today have organizations and processes devoted to software and hardware governance, and these need to be leveraged for the SOA world.

Financial. Budget allocation in a pre-SOA world happened in silos at the project, group, or department level. SOA, on the other hand, is about sharing capabilities as services and leveraging assets across the enterprise. SOA may require new policies and procedures (including chargeback models) for funding services and architecture. Ideally, these policies should facilitate 1) the sharing of hardware and software infrastructure that is the backbone of an enterprise wide SOA; 2) the funding of business and technical services that will be shared across multiple departments; 3) the funding of SOA-related capabilities that aren’t delivered as part of existing projects; and 4) the funding of an active enterprise architecture (EA) group and SOA Center of Excellence—which usually happens when your SOA journey is already well under way. The sooner you balance the interests of service providers, who pay an overwhelmingly large portion of the cost for producing a service, and service consumers, who pay an underwhelmingly small cost for using the service, the more reuse you will get! The greatest benefit and return on investment with SOA occurs when companies stop trying to maximize investments locally (silos) and start maximizing all assets across the entire enterprise. SOA usually costs more at the outset—as the first projects build out the services and infrastructure that will be leveraged later by other areas—but benefits increase as time goes on.

Portfolios. You need to enact policies around three types of portfolios: business and foundational services portfolios, project portfolios, and enterprise applications portfolios. You should 1) ensure that application lifecycles (upgrade, enhancement, maintenance, and retirement) are consistent with your SOA strategy and enterprise architecture—especially with the SOA standards on which interoperability is built; 2) ensure that hardware and software agendas and plans are consistent with your SOA and enterprise strategy; 3) create projects to align applications and infrastructure with the milestones and goals of your SOA Road Map; 4) plan your business services and foundational/technical services portfolios so that you can phase in services that are in sync with projects that will be leveraging them; and 5) ensure that your packaged enterprise applications—which are a potential source of services—are leveraged in your business services portfolio, and that decisions on new packaged applications conform to the SOA strategy and EA direction. At the same time, your policies should ensure that the project you are building using SOA principles contributes services to the business services portfolio, from which other projects can then benefit.

People. SOA is not just a technology shift. Policies to encourage desirable behavior among employees must be part of your SOA governance. Specific areas that need to be considered include 1) assigning and empowering employees who are responsible for driving process improvement - often termed Process Officers (SOA is about improving business processes, thus someone needs to be responsible for making it happen); 2) developing the skills necessary for architecting, building, testing, and deploying services and service-oriented applications; 3) creating incentives to encourage the building of sharable services and the reuse of existing services; 4) forming an enterprise architecture group to drive adoption of EA disciplines and SOA in particular; and 5) creating a group that is specifically tasked with governing the SOA roadmap. Typically, the SOA governance group consists of representatives from EA, the different lines of business and finance.

Projects. SOA has a pronounced impact on how projects are executed. Not all projects are compatible with SOA technologies, and even projects that are designed for SOA must have a range of additional considerations and policies applied to them. Policies need to be put into place to 1) ensure that appropriate projects are selected for the application of SOA techniques and technologies; 2) prioritize projects and align them with the SOA strategy and SOA Road Map (for example, to ensure that projects take into account services that gradually come online as denoted in the business services portfolio plan); 3) address service funding, ownership, and management; 4) drive consistency in service implementation to ensure that shared services are architected and deployed to facilitate and realize sharing (some of these policies will focus on architecture as well); 5) address the creation, storage, and retrieval of shared SOA artifacts; and 6) formalize the governance of the lifecycle of services, business processes, and business rules.

Policies that address the lifecycle of services—or service lifecycle governance—are usually the focus when people talk about SOA governance. Service lifecycle governance covers every aspect of services, including service identification, approval for enterprise services, service design (including interface design, creation guidelines, and best practices), service publishing, change request management, versioning, retirement/sunsetting, deployment, and operations. Proper service lifecycle governance is a critical component of SOA success. Without it, you may have services and SOA technology, but you will not realize the benefits of an enterprise SOA. Technologies such as registries and repositories help to enforce service lifecycle policies, and integrated business process management suites include capabilities to manage the lifecycle of business processes. However, each organization must define and enforce its own best practices to ensure that service lifecycle policies are followed.

Operations. Services that are shared across enterprise departments have operational implications that need to be captured as policies. These policies must address 1) the operational model for services, including who pays for additional resources when services levels are increased; 2) capacity monitoring and planning, to ensure that critical business processes that rely on shared services are monitored and the services that support those business processes have the capacity to handle the load; 3) the handling of policy exceptions and violations; and 4) service execution, including the definition and enforcement of runtime policies such as security, access, logging and billing policies, and service reliability. The runtime enforcement of operational policies is the other aspect of SOA governance that many focus on. A SOA/ Web services management solution can help you apply runtime policies to services.

Six Steps to Successful SOA Governance

So far, we have outlined some of the policies that you should consider putting in place to deliver on your SOA Road Map. We now provide some easy-to-follow rules of thumb (shown in Figure 3) that you should follow when putting together your governance regimes for SOA.

Figure 3: Six Steps to Successful Governance with SOA

Step 1: Define Goals and Strategy. As the very first step, companies need to define their goals and SOA strategy. A company’s SOA strategy must complement and support its goals and business objectives. This point cannot be overstated.

Step 2: Define Policies and Procedures. There is no “one size fits all” SOA strategy, thus there is no single SOA Roadmap, nor a single set of policies and procedures that constitute governance with SOA. For example, if you take a project-by-project approach to SOA, you will tend to focus more on policies that govern technology, project execution, operations, and architectural aspects, and less on governance, organization/people, and financial aspects. (This approach is typical of organizations that don’t have enough senior leadership buy-in early on to enforce cross-departmental issues.) Whatever SOA strategy your organization selects, you should clearly state who has decision and input rights in formulating specific policies, how the policies are communicated, how the execution of those policies is monitored, and how adjustments can be made to the policies based on real-life experience.

Step 3: Define Metrics for Success. Once you know what kinds of policies you need to focus on and where you are in the execution of your SOA strategy, you need to define the success factors and key performance indicators that will let you know you have achieved your goals and objectives. If you do not define metrics to measure the success of your SOA project, you are unlikely to establish the right governance mechanisms, and are equally unlikely to deliver on your SOA strategy. Only when you know what metrics you will be measuring yourself against can you effectively put in place the policies and procedures necessary to achieve the desired behavior and outcomes. Metrics you can use include the number of projects adhering to processes for project execution, the number of reference architectures leveraged, usability of the reference architectures, the number of exceptions and the reasons for each, the number of sharable services created, the number of applications leveraging the shared services, and the cost saved by reusing an existing service. Note that you need to make your goals (which you derive from your metrics) realistic; if you are in the early stages of SOA, then you need realistic reuse goals. The more aggressive these goals are upfront, the more effort and cost you need to invest at the early stages: the appropriate policies; infrastructure, such as registries and repositories; and incentives to change developers’ behavior. Remember, if you don’t know where you’re going, you will never know if you’re going to get there. Measuring your progress is key.

Step 4: Put Governance Mechanisms in Place. Once you have defined your metrics, policies, and procedures, you need to put the governance mechanisms in place—policies, procedures, enforcement, methods of monitoring, exception handling mechanisms, and so on. It is one thing to define these mechanisms, and a completely different matter to implement and enforce them. In order to make the procedures effective, you need endorsement from executives, to “encourage” the behavioral changes that SOA requires and to ensure the participation of the appropriate people. When you are setting up the governance mechanisms, be aware that governance overhead should be commensurate with the stage of SOA adoption and the size of your company. That is, you won’t need much when you’re starting out, but you should plan on progressively phasing in more policies as you increase your SOA maturity.*1

Step 5: Analyze and Improve Processes. You should analyze the results of the governance policies that you put in place and gather metrics on the governance processes themselves. You should also measure the progress that you have made on your SOA Roadmap, relaxing overly restrictive policies where it makes sense and taking corrective action where necessary. A lot of companies separate “policies” (have to follow) from “guidelines” (should follow). Remember, you want to have an open environment in which people communicate their actions and experiences when they go off the beaten path.

Step 6: Refine Your SOA. Periodically, as your SOA matures, you need to create new policies and procedures that enable you to continue to increase your SOA maturity while delivering on your business goals. You will re-evaluate and refine your SOA strategy, along with your SOA goals and objectives, and institute new policies and procedures that enable you to advance on your SOA Roadmap.

Conclusions

The only way to deliver on your business, enterprise architecture, and SOA objectives and increase your organization’s SOA maturity is through appropriate governance. SOA governance is more than just service lifecycle governance or the enforcement of runtime governance policies—it requires policies and procedures that cover every aspect of the SOA project, including organizational, financial, project execution, architectural, operational, and technological. It is critical that as organizations mature, these areas be aligned. We are not suggesting that you put in extensive governance schemes when you start out. Just realize that SOA requires change, and change happens through decisions and policies that are deliberately put into place. However, you must make change happen progressively, incrementally, and ensure that the governance overhead is proportional to the size of your SOA. Finally, remember that without executive-level buy-in, it’s going to be hard to deliver on the full potential of SOA within your enterprise. So make sure you have executive buy-in for enacting cross-departmental governance policies!

In the next article we look at best practices for Governance with SOA.

About the Authors

Mohamad Afshar PhD, is Vice President, Product Management within the Oracle Fusion Middleware development organization. His main focus includes vision, strategy, and architecture, with an emphasis on SOA, EDA and next generation. He works closely with customers advising them on their SOA roadmaps and has spearheaded Oracle's SOA Methodology. Previous to Oracle he was a co-founder of Apama, an algorithmic trading platform (sold to Progress). Mohamad is a frequent speaker at industry events and is a contributing author to business-oriented, technical, and academic journals. He holds a PhD in Parallel Database Systems from Cambridge.

More by Mohamad Afshar

Benjamin Moreland is presently employed at The Hartford as the Director for Foundation Services, which is responsible for the realization of The Hartford’s P&C SOA direction. He has over 20 years of IT experience that includes management, research, presentations and development, with a strong background in artificial intelligence. In the last 3 years at The Hartford, his team has implemented over 80 services, deployed a UDDI registry, SOM, service bus, BPEL and helped set the SOA standards for The Hartford. He has presented at numerous conferences. While at United Technologies and The Univ. of Conn., he led and participated in many research and development efforts. Mr. Moreland has a BA in mathematics, and an MS in Computer Science & Engineering from The Univ. of Conn.

More by Benjamin Moreland