The practice of looking into the future goes back a long way. Once it was done with omens and portents. Now it’s done with streaming databases, neural network algorithms and MapReduce data shuffles. What hasn’t changed is that obtaining reliable and “actionable” predictions remains challenging—especially for businesses.

Outside of capital markets, the pace may be slower. Wall Street has long been in the technology forefront. But event processing and predictive analytics technologies—some people refer to them as strategies—are being tried out in other industries. They are gradually finding mutual use as different types of businesses seek new competitive advantages. Both approaches, along with related “big data” technologies such as Hadoop, may be poised to change how business decisions are made.

Event processing and predictive analytics are “pretty complementary strategies,” according to Neil Ward-Dutton, research director of MWD Advisors, based in the United Kingdom. Beyond capital markets, these strategies can be employed in tandem in online sales and marketing, call centers, telecommunications and other industries, he says. The explosion in Web data is a major driver of greater use of event processing and predictive analytics.

These now-sizzling technologies can be both strange and familiar. Event processing is a unique branch of computing that tracks “streams” of occurrences and looks for underlying patterns. The complex version of event processing—aptly named complex event processing, or CEP— combines data from various sources to form more complicated patterns for processing. The prime example has long been the computer that finds valuable stocks that respond to unexpected market conditions.

Meanwhile, predictive analytics is a methodology that applies specialized algorithms to data sets to create a probability-based predictive model of anticipated activity, not unlike the actuarial tables used to calculate risk and compute auto insurance rates for different types of people.

EVENTS: OUR DIGITAL FOOTPRINTS, OURSELVES
Events have always been there. They just weren’t digital. “People didn’t leave the digital footprints they now do,” Ward-Dutton says. “Now pretty much everything leaves a footprint.” The addition of predictive analytics opens new possibilities for the processing of digital events, he says. But it’s important to connect both event processing and predictive analytics to business objectives.

Tying event technology to operations and analysis results to application integration efforts and useful business activity is important to practitioners of business process management (BPM). Event processing technology is driven largely by a desire for better operations, as indicated by the most recent ebizQ survey of business and IT professionals. A majority of respondents (69%) cited “operational efficiency” as the main goal of their event processing technology efforts.

But analytics has a foothold, too: In ebizQ’s survey, 28% of respondents cited “support for business intelligence/analytics programs” as a primary driver of event processing work. Meanwhile, in the same survey, “integration with other application/systems” (71%) leads the list of challenges to decision management, with “tying analysis to action” placing second (44%) among such challenges.

What are examples of mixed event processing and predictive analytics? The applications and vertical niches can be diverse:
* Manufacturers predicting factory machine failures and preorder replacement parts.

* Software employed to speed up medical device clinical trials for faster time to market.

* Programs predicting the likelihood that a hospitalized infant may contract blood poisoning.

* A financial services company watching website logs and correlating activity to dangerous security events

* An automaker setting up highly automated fraud forecasting filters that better select leasing candidates for scrutiny.

* A travel site that steers users of Apple products to different promotional offers based on the established likelihood that they are willing to spend more on vacations.

In his technology predictions for 2012, Ward-Dutton emphasized the importance of the trend for increased aggregation of core BPM capabilities with complementary platform elements “including event processing, business rules and real-time/predictive analytics.” Understanding events and triggers and deriving insight from combining those event streams with models of, for example, customer segmentations, can be a powerful tool, he says.

CEP: SLOW GOING OUTSIDE OF CAPITAL MARKETS
As organizations of all sizes deal with more Web-created data, CEP will come under consideration. Yet, to date, CEP’s major base remains banking and stock trading. CEP took off in the capital market sector for activities such as algorithmic trading, but it has had some success stories beyond the original constituency, says Philip Howard, research director for Bloor in the United Kingdom. He sees a handful connected by themes.

First, he says, a number of companies have invested in CEP to support BPM and SOA. Second, security information and event management has been a specialized operations niche for CEP. And finally, there is CEP matched with real-time analytics. “My basic view is that event processing is very valuable for predictive analytics, but the market hasn’t taken off yet,” Howard says.

Ward-Dutton adds supply-chain optimization and fleet-logistics optimization to the list of CEP success stories. Where CEP coupled with predictive analytics has shown value is in telecommunications, he says. There, the churn rate—that is, the level of customer attrition—is a crucial issue.

“For a mobile cell company, say, if you can take intelligence and combine it with an understanding of event streams, then you can detect, for example, that a customer has a string of four dropped calls in a week,” he says. “You may know from that customer’s attributes that they are likely to drop their service as a result.”

But with the right software in place, “you can take an action, make an offer,” Ward-Dutton says. “This is where SOA, BPM and events get linked together, too.”

HADOOP, PMML TAKE THE STAGE
Both complex event processing and predictive analytics as they are today require specialized skills. That can greatly add to the expense of implementation and put additional pressure on software architects to get it right.

At the same time, emerging standards show some promise for bringing a bit more commonality to these styles of development. One of these is the Predictive Model Markup Language(PMML), which supports a portable notation that can be used in building predictive models and pattern filters.

A more widely discussed standard is the Apache Hadoop MapReduce software framework. This Java-based distributed file system technique is naturally tailored to run on distributed computer systems. Some might even call Hadoop the poster child for big data.

“Hadoop is really good at embarrassingly parallel problems with massive amounts of data,” says Douglas Moore, principal consultant and architect for Think Big Analytics, a professional services firm based in Mountain View, Calif.

A central premise for Hadoop is reduced cost of software and development. This will appeal to line-of-business leaders, enabling wholly new applications. The framework has already appeared as an adjunct to some CEP engines. And predictive analytics has shown up on Hadoop clusters as well.

Naturally, there are disadvantages, too.

First, it’s still new and raw. That requires developers to build their own tools for many requirements. Perhaps most important: Its newness draws programmers back into data integration in ways not seen for many years. Some will find the programming model restrictive. It can be difficult for developers well-versed in SQL to jump to Hadoop. It can be an even more difficult leap for relational database administrators.

Of course, it’s still early.

“The majority are still testing out Hadoop. Some people are doing it for real,” says Howard. “What worries me is all the emphasis is on Hadoop and people not looking at the alternatives. It’s almost like people are all looking in the Hadoop direction and putting blinders on.”

BONUS CONTENT: NEIL WARD-DUTTON'S CRITICAL FACTORS FOR CEP
Successful matching of business goals and technology know-how is essential for successful matching of event processing and predictive analytics, according to Neil Ward-Dutton, analyst at MWD Advisors in the United Kingdom.

Clearly, this is state-of-the-art technology, and implementation targets should be carefully picked to prevent solving the wrong problem., he says. In addition, sizing the data load is an issue: Too big a helping of big data can be problematic.

“A critical piece of this is getting the domain understanding into the project—not looking at it from just a technology perspective, but looking at it from a business perspective,” Ward-Dutton says.

The challenge is a chicken-or-egg issue, he continues. The technologist and the business analyst have to work together well: “The business analyst says, ‘What if I can reduce customer churn by X%? How do we do that? What information do we have that we can use?’ The analyst does interviews to see what causes people to leave, looks at what causes people to churn,” Ward-Dutton says.

“Then it’s necessary to look at the tradeoffs,” he continued. “If you look at it just from a technology angle, you can look at a lot of crazy ideas.”

The solution, he says: Building great partnerships between people who understand the business and the technology. “The truth is, you may get to the same result using a smaller data set,” he says. “Sometimes the law of diminishing returns sets in.”

READER FEEDBACK: Have you used event processing and predictive analytics technologies together? If so, ebizQ editors would like to hear about your experience. Contact Site Editor Anne Stuart at editor@ebizq.net.



About the Author

Based in Newton, Mass., Jack Vaughan is editor of chief of ebizQ's sister site, SearchSOA.com. Contact him at jvaughan@techtarget.com.

More by Jack Vaughan

-1-

1  

Explore Our Topics

  • EDITOR'S BRIEFING
  • Virtual Conferences
  • Webinars
  • Roundtables

BPM in Action

March 10, 2011

The sixth annual BPM in Action 2011 Virtual Conference will explore cutting-edge market developments in BPM and describe how to leverage them for improved business operation and performance. More

View All Virtual Conferences

Smart Case Management: Why It's So Smart.

Date:Nov 05, 2009
Time:12:00 PM ET- (17:00 GMT)

REGISTER TODAY!

Date:Oct 29, 2009
Time:15:00 PM ET- (19:00 GMT)

REGISTER TODAY!
View All Roundtables
  • Research Library
  • Podcasts
  • News

Joe McKendrick: Part II of II: Designing Evolve-ability into SOA and IT Systems

In part two of Joe McKendrick's recent podcast with Miko Matsumura, chief strategist for Software AG, they talk about how SOA and IT systems need to change and grow and adapt with the organization around it.

Listen Now

Phil Wainewright: Helping Brands Engage with Social Media

Phil Wainewright interviews David Vap, VP of products at RightNow Technologies, and finds out how sharing best practices can help businesses understand how best to engage with online communities.

Listen Now

Peter Schooff: Making Every IT Dollar Result in a Desired Business Outcome: Scott Hebner of IBM Rati

Scott Hebner, Vice President of Marketing and Strategy for IBM Rational, discusses a topic on the top of every company's mind today: getting the most from IT investments.

Listen Now

Jessica Ann Mola: Where Will BI Fit In? Lyndsay Wise Explains

In BI, this tough economy and the increasing role of Web 2.0 and MDM are certainly topics on people's minds today. WiseAnalytics' Lyndsay Wise addresses each of them in this informative podcast.

Listen Now

Dennis Byron: Talking with...Deepak Singh of BPM Provider Adeptia

Deepak Singh, President and CTO of Adeptia, joins ebizQ's Dennis Byron in a podcast that gets its hand around the trend of industry-specific BPM.

Listen Now
More Podcasts
  • Most Popular
  • Quick Guide
  • Most Discussed

Quick Guide: What is BPM?

Learn More

Quick Guide: What is Event Processing?

Smart event processing can help your company run smarter and faster. This comprehensive guide helps you research the basics of complex event processing (CEP) and learn how to get started on the right foot with your CEP project using EDA, RFID, SOA, SCADA and other relevant technologies. Learn More

Quick Guide: What is Enterprise 2.0?

A lot of people are talking about Enterprise 2.0 as being the business application of Web 2.0 technology. However, there's still some debate on exactly what this technology entails, how it applies to today's business models, and which components bring true value. Some use the term Enterprise 2.0 exclusively to describe the use of social networking technologies in the enterprise, while others use it to describe a web economy platform, or the technological framework behind such a platform. Still others say that Enterprise 2.0 is all of these things. Learn More