Which BI Uses Will Finally Drive the Adoption of 'Real-Time' Data Feeds?

Adoption of any technology will be driven faster, always, if there is money involved in its usage directly!

Given that metric, real-time data feeds for Business Intelligence will make sense and get adopted faster if not having them will cost you money or get as much as you thought you might!

Some classic examples are Real-time Stock Ticker Information, Commodities Trading, etc.

On the operational side, service and support functions always had a need for real-time data feeds, especially if there are third parties involved and they are penalized for not meeting service parameters (SLAs) and incentives for meeting them. Then since the real-time feed directly helps or hurts the revenues, adoption will be faster.

This is becoming more common than we think - even large online stores like Amazon or computer makers like HP, Dell use third party manufacturers and shippers. For them real-time BI feeds will be crucial since they promise delivery dates and meeting them will mean customer satisfaction directly!

I agree with Nari.

Furthermore, following on Nari's comments, the question might be rephrased to: "where do real-time data feeds offer the most compelling value in the context of BI?"

It seems that the Complex Event Processing players (CEP) have been addressing markets with that interest for many years: financial, logistics, telecommunications, government security/surveillance, etc. But it seems that the overall pick-up has not been as fast as one might expect. For example retailers with RFID seemed to be the perfect use case some years ago, but has BI for RFID data feeds really taken off yet? Perhaps this is the context that prompted the question of this topic in the first place?

I suspect the "killer app" of "BI for real-time data feeds" might just be sentiment analysis on Twitter data feeds via their API (and fed further by data feeds of other social networking sites). There will be some tricky hurdles to overcome to couple BI with sentiment analysis, given the short amount of text in a Tweet. But if you consider the tweet's source, time, and a few key words, you already have some interesting context to go on. There are already some early players in that space doing some interesting things (eg, Overtone). I think it will really take off in 2010.

I'm not sure why this is a question - the horse has long left on the barn on this question... real-time data feeds have already fundamentally changed the financial services industry, and the federal space / intelligence area. We are well on the way to a real-time fabric and dominant use of CEP technologies.

RFID was hyped several years ago, but event processing / CEP didn't take off in those use cases because the event streams were simply too expensive to generate. RFID tags are still reserved to more expensive items and applications, so the deluge of event data did not happen.

But, no matter, for those of us in the CEP industry, things are going just fine for us in the environments that HAVE events. The rest of the industries will follow over time.

- Mark Palmer, CEO, StreamBase Systems

To Mark's point, if the question was "Which BI Uses will Drive the MAINSTREAM Adoption of Real-Time Data Feeds?", I think the answer is importantly different.

The fact that systems-based event/data streams even within industries are so non-standard, the costs of implementing CEP, BAM and even rules engine technologies is too costly and complex (requiring money, time and scarce systems integrator cycles). What if a new layer of "web services" could help standardize system-generated events, first, within a specific industry and, second, across industries? Wouldn't this lower the barriers to and therefore make more cost-effective the use-cases for using real-time data?

Think of: discrete and process manufacturing, energy/utilities, even retail/distribution and healthcare (where business profit margins are notoriously low) . . . they all stand to gain huge benefit from real-time data and intelligent events for improved performance management (system to system and system to human).

Just a thought . . .

Brian Gentile
Jaspersoft