Data, you may not realize it sometimes, but we do it all for you. Honest.
In a new posting here at ebizQ, Oracle's Jeff Pollack says the time has come to bring business intelligence, data warehousing, and master data management (MDM) into the SOA house. Or bring SOA into their houses.
Jeff, an executive on the Fusion side of Oracle, and author of the enterprise software book "Adaptive Information“ (John Wiley & Sons), points out that vendors, analysts and enterprises alike often get caught up in the software side of things for too long. "In the past 15 years, with the rise of Java, the hype surrounding EII, EAI and SOA, and the rise of XML, and quietly, the billions spent in ETL projects -- it's too easy to forget why we build and buy all that infrastructure. We do it for the data."
The problem is, many SOA projects may not be ready to handle all the data corporations are churning out these days. Most, if not all, operate within a Java Virtual Machine container and can't effectively handle large data loads. Plus, XML can be a cumbersome markup language for data.
Thus, data managers keep relying on their relational databases to do what needs to be done. As Jeff puts it: "Most new XML-centric solutions for data can't scale to the mult-terabyte sized problem that is typical of a Global 2000 business. Thus, a knowledgeable architect will revert to the proven patterns of RDBMS as the backbone of a data architecture using MQ, TPS, and ETL interfaces as the pipes for pushing all that data around."
That's the challenge going forward, Jeff says. SOA still suffers from "poor bulk data transformation performance and inefficiency."
To address this problem, Jeff recommends the creation of "SOA Data Services," which are decoupled endpoints that expose components for working on all types of data. Interestingly, he points out, such services can be created without SOA, and such services have been around for years now, in EDI and Java applications.
Start simple, Jeff advises. A Data Service could simply be a "boring Web Service with simple data actions, or thin SOA façades for wrapping conventional data technology." And remember, it isn;t just about XML. Data Services created these days should encompass different data delivery patterns and formats, Jeff says, noting that "SOA pundits often assume that XML is the only desirable delivery format, but for a data solution to be truly useful for the enterprise, it must support several different delivery styles."
Joe,
Jeff made a very needed attempt to bring some clarity to the confusion about what data integration really means in an SOA. Unfortunately, he was very successful in conveying his thoughts on the subject. I think Loraine Lawson was confused by Jeff's article in here post at http://www.itbusinessedge.com/blogs/mia/?p=328. You have done a better job in your summary above of extracting the meaning. Jeff argues that data services are critical to SOA, but not all data movement and manipulation can be done within those data services. Instead, data services will in most cases provide control or access layers for manipulations and movements done using non-services-based infrastructure, like DBMS and TPS. Providing such a "services facade" on existing and appropriate data infrastructure is what we have been doing with the XAware.org project. This makes a lot of sense to us and we think, if we are reading him right, that Jeff agrees. Bill Miller, www.xaware.org
Joe,
Jeff addresses an important topic with his recent article. Thanks for reviewing it here. You did a nice job of summarizing the key point - that SOA infrastructure (e.g. app server container) isn't the place to move or manipulate large data sets. Instead, for large data sets, SOA data services should provide access control and an XML "thin facade" for data that continues to be stored, manipulated, and moved by exisitng data infrastructure. At XAware.org our open source project provides a data services approach that does just that. Bill Miller, www.xaware.org