Integration on the Edge: Data Explosion & Next-Gen Integration

Hollis Tibbetts

Application Integration in the Cloud - Finally Here

user-pic
Vote 1 Vote

Cloud-based Data Integration has been available for quite some time. But Cloud-based Application Integration (Cloud-based EAI) is a new arrival.

Vendors like SnapLogic and Boomi helped lead the charge into that Cloud-based Data Integration space, and heavyweights such as Informatica helped make it a mainstream technology.

The First Cloud-based Application Integration Platform

MuleSoft, a well-known vendor of open-source message bus software, is poised to change the Application Integration (EAI) landscape as well as the Cloud Integration landscape with its new iON Cloud-based Application Integration platform. MuleSoft's iON is the first full-function built-for-the Cloud Application Integration suite.

Data vs. Application Integration

Data Integration, which is a "batch mode" or scheduled operation, is about synchronizing data between different data sources and targets. It can involve a lot of data manipulation, reconciliation, de-duplication, cleansing, standardization and other data-intensive operations. It fundamentally provides a layer of abstraction from the underlying data sources. A Data Integration task might run once a week, once a day, once an hour, or a few times an hour, or even more often than that. It doesn't run hundreds or thousands of times an hour, though. A Data Integration task may involve a single record of data or Terabytes of data.

Application Integration is about the reliable and timely exchange of messages between applications (rather than data sources). It typically operates at near real-time speeds, and often includes workflow types of functionality on top of the core message-level guaranteed delivery capabilities. Application Integration is about enabling business processes that span multiple different systems, and provides a level of abstraction from the underlying applications and business processes. Typically the amount of data involved in a single Application Integration task is quite small (although there may be many thousands of Application Integration tasks that get processed in the course of an hour).

The following diagram (click for full-size image) shows a high-level depiction of the differences between the two different types of integration. Whereas Data Integration is about the movement and transformation of data from one place to another - ignoring application or business logic, Application Integration is about linking systems together at the application logic level - the data simply goes along as a spectator.app vs data integration.png

Deciding - Which Integration Method for You

As I've written before, neither method of integration is superior or inferior to the other - it simply depends on the need. For example, if you need to populate a large data warehouse from numerous transactional systems, Data Integration is probably the way to go. If you want to create a single business process that spans multiple independent applications, then Application Integration is the way to go. Nearly all enterprises with integration needs require BOTH types of integration technology.

Application Integration Example

As a way of explaining the benefits of Cloud-based Application Integration, it may be most effective to present it in the context of a common use case that most people can relate to - the "order to cash" cycle.

Order to Cash (OTC) is a common business process - every business that exchanges some sort of good for some sort of compensation has a version of this business process. Consider a scenario where a business sells physical goods to other businesses. They receive orders for goods from customers. They process orders, create invoices, ship orders, receive payments. This process spans potentially multiple different systems - which may span multiple organizations (as many companies outsource shipping). A high level diagram of this process might look similar to the following diagram (click for full-size image):
OTC.png

For a business process such as OTC, which is a single logical business process, Application Integration rather than Data Integration is the preferred approach, for a number of reasons:

1) Timing - Application Integration is event-driven and near realtime (rather than scheduled)
a. The faster the process completes, the faster the company gets paid - resulting in lower Days Sales Outstanding, better cashflow.
b. The faster and more accurately the orders ship, the higher the level of customer satisfaction

2) Integrity of the process is important. Application Integration frameworks guarantee that certain events will happen - and that they will happen in the correct order.
a. The order that the events occur in is important. For example, an order should not ship without an invoice being created. Orders always ship with the product and an invoice together.
b. As soon as a payment is received, financial data in multiple places needs to all updated "together" - customer records, corporate ledgers, etc.

Cloud-based Application Integration Will Change EAI

MuleSoft's iON marks the beginning of a tremendous and long-overdue shift in the Application Integration market, for at least three major reasons:

First, on-premises EAI platforms have been around for a long time - they've been mainstream since the late 1990's. They're highly capable, but have a reputation as being very complicated and expensive to implement.

Pretty much any enterprise-class software that ends up in the Cloud becomes tremendously easier to use and faster to implement. Compare Siebel on-premises vs. Salesforce's SaaS application for example.

It's not that the Cloud magically makes things better - it's that the software packages built for the Cloud are new, more cleanly architected, free of baggage and legacy problems. For example, many on-premises integration stacks have code dating back to the early 1990's, with user interfaces that would easily be at home in a Windows 95 environment.

They've been extended dozens of times and have become unwieldy, unproductive.
Secondly, increasingly more and more operational as well as analytical applications in the enterprise are moving to the Cloud. SaaS applications like NetSuite or Salesforce, and increasingly, Cloud-based analytics platforms are the default choice for those doing significant analysis and number crunching.

With the locus of applications and data moving increasingly towards the Cloud, that's where Integration increasingly belongs. Just as SaaS and Cloud are becoming not only mainstream - but preferred choices for business applications, Cloud-based Application Integration will become the preferred choice for EAI.

Finally, MuleSoft's iON follows a SaaS pricing model. Integration products like TIBCO can easily cost hundreds of thousands of dollars - typically payable up-front, with yearly maintenance easily costing tens of thousands of dollars. MuleSoft's iON has multiple tiers of pricing ranging from $150 a month to $1,000 a month and a pay-as-you-go model.

Much like Salesforce with its $25 per month entry-level cost upset the Siebel Systems (whose competing product was often hundreds of thousands of dollars) applecart, MuleSoft's iON will do the same to the EAI marketplace.


1 TrackBack

TrackBack URL: http://www.ebizq.net/MT4/mt-tb.cgi/18257

Cloud-based Application Integration Gains Momentum with "iApps" from Integration on the Edge: Data Explosion & Next-Gen Integration on January 19, 2012 1:15 PM

Real-time application integration is picking up speed in the Cloud.  MuleSoft and Zuora just made a significant announcement on this front - one that validates the need for Cloud-based Application Integration, and shows that market is gaining... Read More

2 Comments

You missed the inventor of the space who combined the ETL, EAI and EII Cloud Integration market over 6 years ago and has by far the most Enterprise Cloud Application Integration References in the SaaS market, offering not just a simple Public Cloud Application and Data Integration capability, but also Physical and Virtual Appliances that can sit behind a business partners or the clients firewall and download integration assets built in a Public Cloud.

Cast Iron has now been part of IBM WebSphere (the leader of OnPremise Application integration) since May of 2010, and is now enabled in every major country and language in the world. It also has the Global WebSphere organization focused on this market and another 1000+ local business partners throughout the world becoming certified resellers of IBM WebSphere Cast Iron.

Informatica (Historically Data Integration Company), Boomi (Previously EDI Company) decided to build a serious product for this market 3 years ago, Tibco and Snap 2 years ago, Mule (EAI) this year.

You may want to broaden your research in this space as it is evolving quickly, the legacy definitions and approaches of the application and data integration 1.0 market are struggling to keep up with this new Cloud Application and Hybrid Integration 2.0 market.

It is becoming harder and harder for the smaller integration players to invest appropriately in this space outside the US and provide the necessary support for the Cloud Application connectivity layer around the world, especially as Cloud Application customers continue to demand their data, application and user integration comes from one stable vendor who can handle global hybrid integration projects in different countries with unique data compliance and support requirements.

but is there still such a market for EAI?

This blog offers an informed and informative perspective on the ongoing explosion of data and the technologies used to turn this data explosion into assets and competitive advantages.

Hollis Tibbetts

Hollis has established himself as a successful software marketing and technology expert. His various strategy, marketing and technology articles are read nearly 50,000 times a month. He is currently Director for Global Marketing Operations for Dell Software Group. Hollis has developed substantial expertise in middleware, SaaS, Cloud, data management and distributed application technologies, with over 20 years experience in marketing, technical, product management, product marketing and business development roles at leading companies in such as Pervasive, Aruna (acquired by Progress Software), Sybase (now SAP), webMethods (now Software AG), M7 Corporation (acquired by BEA/Oracle), OnDisplay (acquired by Vignette) and KIVA Software (acquired by Netscape). He has established himself as an industry expert, having authored a large number of technology white papers, as well as published media articles and book contributions. Hollis is a top-ranked author on Sys-Con media, is also published on Social Media Today "The World's Best Thinkers on Social Media", and maintains a blog focused on creating great software: Software Marketing 2013. He tweets actively as @SoftwareHollis Additional information is available at HollisTibbetts.com All opinions expressed in the author's articles are his own personal opinions vs. those of his employer.

Recently Commented On

Monthly Archives

Blogs

ADVERTISEMENT