Creating an Integration Competency Center and Keeping It Running

The increasing complexity and number of application integration (AI) projects can no longer be dealt with on a project-by-project basis. The wide scope of AI projects requires the use of more specialized organizational structures and competencies.

In addition, the diverse and rare skills required (for example, data transformation and reconciliation) tend to be spread across disparate parts of the organization, which hinders many companies' attempts to take a consistent and integrated approach. Creating a single, shared service center that brings the diverse skill set together helps to create the focus, momentum, critical mass, processes and standards required to build and execute a world-class AI strategy.

By reviewing and contributing to AI projects, Gartner has outlined an ordered, managed and sound evolution path toward a mature approach to AI. Following this approach, stage by stage, is a best practice, but it is not the only method that works. Each stage builds on the last, and skipping a stage could be dangerous to the integration initiative. The structure and the nature of the working group changes at each stage. At stage one, most integration efforts are carried out by a few individuals, sometimes belonging to some sort of architecture or integration group within the IT organization. At stage two, the integration team begins to form, often spinning out of a general architecture group. By the time stage four has been achieved, integration technology is pervasive throughout the IT organization and the company, and the ICC has become a mature service center within the IT organization.

Following all four stages of Gartner's approach sequentially ensures an optimal execution of a corporate-wide integration strategy.

Stage One: Plan and Justify

In most cases, one of the painful symptoms of a company's problems at this stage is IT's inability to address specific business needs that require AI, such as the elusive "single view of the customer." Often, this is caused by a lack of IT flexibility. As a result, the business pressure increases: even if the IT organization realizes the technical potential of AI as a solution, it normally struggles to articulate the business value of a systematic, explicitly planned approach to AI in a business case. When the business value is articulated, communicated and accepted, and a senior business sponsor is identified, companies proceed to the selection of one or more AI vendors.

Many leading vendors offer similar integration functionality, but AI requirements are so diverse that selecting the appropriate vendor continues to be a key success factor for the integration efforts. In particular, Gartner found that structured, balanced vendor selection processes -- such as the analytical hierarchy process -- followed by simple, but thorough, proof of concepts with the shortlisted vendors, produce the best results.

Many AI initiatives reach a deadlock at this point. The most common reasons include an unsatisfying requirements fit with the technology, pre-existing political alliances with installed integration vendors, projected integration costs being too high or the evaporation of business sponsorship. In most cases, the only solution is to start again at the beginning of stage one.

In addition, many organizations make default decisions on the integration technology (for example, because they already have some integration technology or because they have an established relationship with a vendor that provides an integration product). However, organizations should avoid rushing into this kind of decision without evaluating alternative technologies, because it seriously affects the effectiveness of the Application Integration projects and it makes the risk of failure unnecessarily high.

Stage Two: Pilot Projects

During stage one, most integration efforts are carried out by a few individuals that belong to an architecture or integration group in the IT organization. These resources have championed AI from the beginning, developed the business case, selected a technology vendor, supported the proof of concept, and they form the core of the team that will evolve into an ICC. At stage two, it becomes clear that the initial resources will not be sufficient to run a project that is likely to cover the needs of several business units and last for months. The project team needs to be officially established so that it has more visibility, responsibility and accountability in the IT organization.

At this point, some companies begin to execute the AI projects that sparked the beginning of stage one, while others decide to bring the proof of concept one step forward by testing the AI technology thoroughly on a practical scenario. Selecting the appropriate pilot project is the next hurdle. A pilot project that is too simple will not prove the business benefits fully and will cause skepticism about the business value of AI to the company. A pilot project that is too difficult may fail, which will seriously undermine the credibility of the business case and the business sponsorship. Apart from these simple considerations, choosing a pilot project is far from trivial, and needs to be analyzed very carefully.

Companies cannot make the transition from stage one to stage two without building a project team that contains resources with multiple skills. In most cases, the more experienced and knowledgeable resources within the original architecture or integration group tend to focus on the overall architecture of the project, while the resources that are more junior and more accustomed to technology start to get familiar with the details of the chosen integration middleware technology. Normally, these resources are augmented with a small project team for the pilot project, either sourced from the technology vendor (or one of its partner system integrators), or seconded from different parts of the IT organization.

The resulting team is the very beginning of the ICC.

This diverse team (often a temporary task force) executes the pilot project. The criteria for success must be set and agreed with the business sponsor at the beginning. Simple, business-oriented numerical criteria work best (for example, "it takes n days to execute this process before the pilot; after the pilot it will take n/2 days"). These criteria will form the basis for the metrics used to measure the impact of the ICC.

In most cases, a successful pilot project is moved into production, where it proves that AI works and delivers business benefits. If the pilot project fails, it is imperative to run a post-project causal analysis review to analyze the reasons for the failure and learn from any mistakes made. Unfortunately, a failed pilot project will damage business confidence in AI, and the recovery from this setback is long and difficult. However, the need for AI has not disappeared, and it will have to be addressed - possibly by a different technology. The temptation to continue using point-to-point integration as a default integration strategy will be stronger than ever, but should be avoided.

Stage Three: Early Adoption and Proliferation

The transition from stage two to stage three requires the creation of a permanent application ICC, which helps initially by providing encouragement and technical skills to multiple project teams, and will evolve into a shared services organization for projects that require integration. In addition, the ICC should be the permanent custodian of integration-related metadata, such as XML schemas and Web Services Description Language (see, "Best Practices: Managing Application Integration Metadata," TG-15-1933). The ICC will be successful only if the rest of the IT organization and the business units buy into it. The ICC could be rejected if it is perceived as a threat, constraint or rival. The architecture group will continue to have strong ties to the ICC.

At this point, the emphasis should shift from technology to organization; the technology is proven, but needs to be applied throughout the organization. The benefits of AI need to be communicated continuously, possibly through the vehicle of an enterprise nervous system or an enterprise architecture, both within IT and throughout the various business units. In addition, companies at this stage identify the first process improvements and begin consolidating best practices.

Stage three is when the AI initiative (and the ICC) emerges from the shell of the first projects, and begins to expand its visibility, impact and benefit by stretching its influence to the whole company, particularly large organizations. Many large and multinational organizations have multiple integration projects or multiple ICCs in different geographies or business units. This is particularly likely when different business units operate autonomously in different markets, such as in large conglomerates, and share the same needs without being aware of it. The spheres of influence of two or more ICCs are likely to overlap because of the nature of AI. In addition to the usual conflicts between proponents of different technologies, this overlap always poses fundamental governance problems that go far beyond the IT organization's control.

This problem has no quick or easy solution. In most cases, the overlapping and competing integration initiatives slow down, waiting for corporate governance mechanisms (if they exist) to take effect in the IT department. However, most governance issues take months to clear. Business sponsorship is crucial in tackling this problem, because it can accelerate decisions through the organizational chain. The ideal situation is to have one master ICC that rules the others, where necessary, to avoid duplication of functions and resources, the use of different methodologies and integration platforms, and stalled decisions based on unclear governance policies. The lack of a master ICC or of a clear federated operational model between the various ICCs can dilute the benefits of the ICC.

In midsize organizations, one ICC should succeed, operate as the service center for the whole organization and progress to stage four. In loosely coupled conglomerates or large organizations, multiple ICCs (for example, for separate lines of business or major geographies) will develop independently, and should operate cooperatively (for example, using a federated model with clear governance policies). The governance rules used to make decisions between the ICCs should mirror corporate governance rules as much as possible. For example, if corporate company-wide organizational units are involved deeply in the decisions made in the business units, the corporate ICC should rule and be prescriptive toward the ICCs in the business units.

Stage Four: Broad Adoption -- Toward Maturity

Few organizations have reached stage four nowadays. At this stage, AI is widespread in the company, and the ICC operates under well-defined rules of engagement. In addition, the business value of AI is well communicated and used consistently by the business. In most cases, the key issue is how to measure this value. Mature ICCs have developed elaborate metrics to assess how they perform within the organization. ICCs have to prove continuously that they deliver value because most companies perceive them as an additional cost. The business need for AI can be satisfied without an ICC, but then the duplication of skills and resources tends to be significant. Therefore, ICCs save money rather than generate additional IT costs.

There is no right or wrong size for an ICC -- Gartner has seen highly sourced or outsourced ICCs of half a full-time equivalent, and complex teams of 150. In most cases, the ICC's size is related to the complexity of the population of applications, the number of AI projects running, the sourcing culture within the organization, the depth of the integration needs and the company size.

The placement of the ICC within (or even outside) the IT organization is another important issue. Many ICCs report to a head of application development, while others are aligned with other competency centers within a technical services group. Because of its central role in delivering business value, its continuous need for sponsorship, and the necessity to communicate with other IT groups and the business, an ICC should report to a CIO, or have plenty of CIO visibility if it is placed elsewhere in the IT organization.

The Bottom Line

Creating an integration competency center, and keeping it running, is not a simple task. By assessing their level of maturity and following a disciplined development path for their application integration needs through the four stages outlined by Gartner, organizations will form an ICC naturally and incrementally, and will be able to get the most business value from integrated applications.

About the Author

Paolo Malinverno is a vice president in Gartner Research. Mr. Malinverno covers a broad set of application integration and middleware areas, including integration organizational issues, integration competency center, enterprise architecture, application integration, middleware, B2B and collaborative commerce.

Mr. Malinverno is a 25-year IT veteran, and his experience ranges across a wide variety of technology platforms. Prior to joining Gartner, Mr. Malinverno was director of product management and principal consultant for a leading messaging and directory technology provider. In this role, he developed market and technology strategies for the whole product range of the company and was instrumental in developing strategic directions for diverse environments, with many new and emerging technologies.

Mr. Malinverno holds a degree in computer science from the University of Milan, is fluent in English and Italian, and reads business French, Spanish and Portuguese.

More by Paolo Malinverno

About Gartner

Gartner is a research and advisory firm that helps more than 10,000 clients leverage technology to achieve business success. GartnerŐs businesses consist of Research, Consulting, Measurement, Events and Executive Programs. Founded in 1979, Gartner is headquartered in Stamford, Connecticut and has over 3,800 associates, including approximately 1,000 research analysts and consultants, in more than 75 locations worldwide.