Integration on the Edge: Data Explosion & Next-Gen Integration

I took advantage of an opportunity to listen in on a panel discussion sponsored by Data Integration heavyweight Informatica. Hosted by THINKstrategies' Jeff Kaplan, with panelists from Astadia, D&B, Informatica and PSA Financial Services, this interactive discussion reviewed the state of Cloud Data Integration today and made some speculations for 2012 and beyond.

I'd like to share some of the points raised during this panel discussion, mixed in with some of my own thoughts as well. The panel discussion and the slides will be available shortly at informaticacloud.com/news/webinars.html

As I've written many times before, data and applications are increasingly moving to the Cloud - and that's increasingly where Integration belongs. Although existing on-premises applications and data sources aren't suddenly disappearing en-masse, deployments of new business applications and new analytics platforms are heavily weighted towards SaaS. Additionally many new sources of data such as Social Media are entirely Cloud-based.

The "locus of integration" is migrating rapidly from the on-premises data center to the Cloud - and unless there is a compelling reason otherwise, the default choice for any new integration solution should be "Cloud-based".

As evidence of this shift, the panel discussion started with a few data points:

• 80% of new software offerings will be available as cloud services.

• By 2014, over one-third of software purchases will be via the cloud.

• Global cloud computing traffic will increase twelve-fold, from 130 exabytes today to 1.6 zettabytes in 2015, a 66% CAGR. To get a perspective on this, the universe is only about 1 "exa-seconds" old. An "exa" is a very big number, and a "zetta" is 1,000 times bigger.

• Cloud represents 11 percent of data center traffic today, growing to more than 33 percent of the total by 2015.

As further indications of the need for better Data Integration strategies - and as a good backdrop for this discussion:

• IDC points out that the "digital universe" of information and content will expand by almost 50% in 2011 -- to almost 2 trillion gigabytes.

• Gartner Research believes that over 85 percent of Fortune 500 organizations will fail to effectively exploit big data for competitive advantage - even three years from now (which likely means things are even worse than that now).

• Nearly every organization of every size is struggling with the challenges of cost-effectively integrating applications, systems and data sources.

Jeff Kaplan (THINKstrategies) posits that 88% of SaaS/Cloud companies identify integration as important in winning new customers and a common sales hurdle. 63% believe integration is a critical part of their solution, but 80% say integration is too time-consuming. It's not only impacting the end-users of SaaS/Cloud applications, but also the SaaS/Cloud vendors themselves. I've written extensively on the need for all business application vendors to incorporate integration into their platforms in "Software Vendors - Integrate or Else"

Cloud-based integration solutions are the appropriate answer for these (not to mention many other) challenges.

Compared to old-fashioned on-premises solutions, Cloud-based Integration:

• Eliminates much of the software and system procurement, deployment and management challenges.

• Accelerates the implementation process and reduce staff administration requirements.

• Reduces the costs and risks associated with adopting new business applications.

• Increases data quality, user productivity, collaboration and performance measurement capabilities.

Informatica would know a thing or two about the locus of integration moving to the Cloud. Their Cloud Integration Platform already processes over 20 Billion transactions and 3 million integration jobs a month-a staggering number, really.

My prediction for the new year - Informatica's Cloud Integration Platform to hit 40 Billion transactions and 6 million integration jobs a month by December 2012, a 100% growth.