It must be the time of the year to introduce new acronyms (or maybe I am just noticing more that have been around for a while). Oracle is touting Extreme Transaction Processing (XTP) as a technology capable of dealing with the ever increasing data volumes which are predicted in most industries. The prize market is actually financial services where, as I previously wrote about in my Lustratus blog, the volume of market data is predicted to increase by 900% by 2012.

The business needs associated with this data deluge are not only storage – and in some cases storage is not relevant as the data has only transitory value. The greater value is in identifying the anomalies or unusual combinations that may correspond to the financial opportunity or a potential fraud. Oracle is clearly trying to address both the traditional transactional processing requirements and these new requirements which are more usually solved with Complex Event Processing tools.

As Oracle’s Dave Chappell puts it, XTP allows “transactions to occur in memory and not against the backend systems directly due to the need for extremely fast response rates, but still including transactional integrity” Fair enough.

However, he goes on to say something that Streambase and Apama (two CEP engine vendors) among others may be surprised to hear:

“Usually, a complex event processing engine is something that can also capture, correlate and apply decision rules to look for specific patterns in events over time and look for exceptions. However, there are certain applications that operate on streams of event data that is so voluminous that typical backend solutions or storage solutions just can't handle it.”

The whole purpose of CEP engines is to provide real-time analysis of massive event streams. My understanding is that to do this they precisely by avoiding using the backend solutions which Dave refers to. Which makes Dave's remarks rather confusing.

Stepping back however, it is clear that this announcement further demonstrates the growing interest among vendors in the potential value of the data deluge problem being solved. Whether the demand for these solutions will grow and break out of its traditional niches (algorithmic trading in financial services and some specialised security applications) is not yet clear.

Ronan

Posted by rbradley in CEP • Financial Services • Product news | Permalink | Comments (0) | TrackBacks (0)

When everybody starts to agree that something is bound to be the next big thing, I get jumpy. When people start ramming the Business word in to create a new acronym, I get really jumpy. Complex Event Processing has been getting hype for a while and now IBM (according to Joe) has committed the crime:

“IBM wants to take the technical allusions out of the term itself, referring to it as “Business Event Processing.” The renaming to BEP makes sense, since I doubt if a concept that starts with the word “complex” is going to win a lot of converts.”

However, I don’t disagree with sentiment that CEP could be the next big thing. It has been considered and used a little to deal with the general increase of data volumes in some industries (financial services being in the “lead” as I covered here ).

However, there is potentially much more opportunity in the SOA context as IBM seem to recognize. As the volume of network traffic related to SOA increases, the need increases to monitor, manage and react to anomalies. Anomalies which could correspond to simple human error, to unexpected loads and even to illegal or fraudulent behaviour. Detecting an anomaly may require looking into an event, detecting specific sequences of events or any arbitary combinations of the above. CEP provides a good approach to detecting subtle anomalies occurring in massive message flows in real-time. While you could have used CEP (if it had existed) in the pre-SOA world, it would not have made much business sense: SOA potentially divides many more business processes between applications (into the SOA layer) and also increases the total volume of messages. The combination makes the total value of the messages in the SOA layer significant and therefore controlling the SOA layer becomes as important as managing applications.

The caveat is how quickly and in how many organisations if at all will this level of SOA deployment occur. I remember talking to a senior architect in a US bank last year who was pro-SOA but would never consider shifting the centre of gravity of business processes out of applications. Hence he would never need the sort of CEP control layer capability that I was proposing.

Time will tell – as with most “next big things”, it will take longer and be less massive but CEP will certainly be discussed and used in increasing levels throughout 2008 and beyond. And no, I don't think many people will call it BEP any time soon.

Ronan

Posted by rbradley in CEP • Mash ups | Permalink | Comments (0) | TrackBacks (0)