Integration on the Edge: Data Explosion & Next-Gen Integration

Hollis Tibbetts

Confusion in the Cloud - Application vs. Data Integration

user-pic
Vote 0 Votes

'Cloud Application Integration' is often used interchangeably with "Cloud-based Data Integration".

Even for on-premises software, "Data Integration" and "Application Integration" are commonly considered synonymous - many use the terms "interchangeably without understanding the significant differences between the two - even though these two categories of software have been in existence for years and years.

Confusion is rampant - in many cases, this confusion has been exacerbated by software vendors on both sides of the equation attempting to broaden the appeal of their products.

They are two very different types of integration with separate approaches, methodologies and tools. Each has its role, one for managing transactions and one for managing the data that the transactions operate on. It's critical that you know which approach is right for you.


DI and EAI - Two Different Animals
For well over a decade, "application integration" (sometimes referred to as EAI) and "data integration" (or DI) have been distinctly different disciplines. Although some analyst groups predicted that the two categories of products would merge, that is not much closer to happening now than it was ten years ago. The fundamental architectures of these two classes of products are light-years apart.

In the on-premises data center, organizations determine very early on that they need a product like TIBCO or webMethods versus a product like Informatica. It's rare that webMethods would ever compete directly against Informatica for a particular piece of business. When it happens, it's often the case that the organization is trying to use one product or the other inappropriately.

Application Integration
Fundamentally, Application Integration links multiple applications together at the functional level. It deals with things at the transactional or service-call level. It is "aware" of how different pieces of information (such as the creation of a new customer) come together to create one atomic unit that cannot be subdivided without introducing data inconsistencies (i.e. "corruption") into the system.

With Application Integration, a transaction, such as the creation of a new customer, moves together as one unit - even if it is decomposed into multiple different pieces at the final destination. For example, creating a customer in one application may involve at least three "bits" of information - Corporate information as well as multiple addresses (bill to, ship to - for example). In the destination application, that information is likely to be stored in different locations (a "company" table, a "ship to" table and a "bill to" table). It would make no sense to copy over the "ship to" information without the "bill to" information. That would create a form of data corruption.

Furthermore Application Integration systems are typically event-based in nature - that is to say, as soon as "something happens" in the originating application, like the creation of a new customer, the transaction is sent over to the destination application(s). The goal is to do this in "real-time", so that multiple systems are always synchronized.

Many Application Integration solutions have their roots in message-queuing or "enterprise service bus" products - for example IBM's MQ Series - which explains their orientation around the concepts of real-time performance, transactionality, reliability, recoverability and once-and-only-once guaranteed delivery.

A typical use case for this type of integration software - as soon as a customer order is placed, generate an invoice and update the financials database. Time is money.

Data Integration
Data Integration, true to its name, deals with the synchronization, standardization, transformation, mapping, quality and transport of data - potentially large amounts of it - between different systems. It is unaware of the concept of a transaction. These products in most cases have their conceptual roots, if not their actual roots, in ETL (Extract, Transform & Load) software. They represent an evolution - albeit an significant one, of ETL technology.

With Data Integration, the physical data representation is abstracted, as is the access and transport of the data. That makes it quite simple to create "maps" between different data sources (for example how a customer order in one system "maps" to a customer order in another system), as well as to introduce things like transformations and validations.


Which is Better?

Data Integration is neither superior to, nor inferior to Application Integration. It is simply different. A 7-seat SUV is neither better than, nor worse than a 2-seat sports car. There are simply some situations where a sports car is called for - and others where a 7-seat SUV is in order.

When transactions span multiple systems or applications, Application Integration is the way to go. "Analytics" is traditionally Data Integration; "Operational" is traditionally Application Integration.

Cloud Confusion with Integration
With Cloud-based integration - i.e. where the integration stack is cloud-based, confusion is even more prevalent between Data Integration and Application Integration than it is with on-premises software.

In the Cloud, Application Integration and Cloud-based Data Integration are confused for two simple reasons:

1) The two terms are confused everywhere else, so why not on the Cloud too?
2) Cloud-based Application Integration didn't really exist - UNTIL NOW.

A New Category of Cloud Integration Products
A new category of Cloud Integration solutions is emerging - Cloud-based Application Integration.

This category is being lead by a well-known name in open-source Enterprise Service Bus software - MuleSoft. Their iON product is the first cloud-based integration platform for Application Integration, rather than Data Integration.

The evolution of this category of software is further evidence of the growth and maturation of the SaaS/Cloud software market. And products such as MuleSoft's iON are enabling a new class of integration that is sure to benefit SaaS/Cloud customers and vendors alike.

1 TrackBack

TrackBack URL: http://www.ebizq.net/MT4/mt-tb.cgi/18245

Cloud-based Application Integration Gains Momentum with "iApps" from Integration on the Edge: Data Explosion & Next-Gen Integration on January 19, 2012 1:15 PM

Real-time application integration is picking up speed in the Cloud.  MuleSoft and Zuora just made a significant announcement on this front - one that validates the need for Cloud-based Application Integration, and shows that market is gaining... Read More

This blog offers an informed and informative perspective on the ongoing explosion of data and the technologies used to turn this data explosion into assets and competitive advantages.

Hollis Tibbetts

Hollis has established himself as a successful software marketing and technology expert. His various strategy, marketing and technology articles are read nearly 50,000 times a month. He is currently Director for Global Marketing Operations for Dell Software Group. Hollis has developed substantial expertise in middleware, SaaS, Cloud, data management and distributed application technologies, with over 20 years experience in marketing, technical, product management, product marketing and business development roles at leading companies in such as Pervasive, Aruna (acquired by Progress Software), Sybase (now SAP), webMethods (now Software AG), M7 Corporation (acquired by BEA/Oracle), OnDisplay (acquired by Vignette) and KIVA Software (acquired by Netscape). He has established himself as an industry expert, having authored a large number of technology white papers, as well as published media articles and book contributions. Hollis is a top-ranked author on Sys-Con media, is also published on Social Media Today "The World's Best Thinkers on Social Media", and maintains a blog focused on creating great software: Software Marketing 2013. He tweets actively as @SoftwareHollis Additional information is available at HollisTibbetts.com All opinions expressed in the author's articles are his own personal opinions vs. those of his employer.

Recently Commented On

Monthly Archives

Blogs

ADVERTISEMENT