Editor’s Note: In this Q & A, Forrester Research Principal Analyst Mike Gualtieri speaks with ebizQ’s Peter Schooff about the growing convergence of complex event processing (CEP) and business activity monitoring (BAM). Their conversation, excerpted from an accompanying podcast, has been edited for length, clarity and editorial style.

ebizQ: From my perspective, there's been a lot of talk about both complex event processing and business activity monitoring and business activity monitoring. How do these two technologies complement each other?

Gualtieri:
I think you're right to say that they complement each other because business activity monitoring is about monitoring the business, just like it sounds. Complex event processing is a lower-level technology that can support that.

So imagine you're trying to monitor business activity like transactions going through an e-commerce system or deliveries if you're a large logistics company. There are certain events that may be notable in those business event streams, [things] that are happening that you want to take action on. CEP is an underlying technology that can--in some cases but not all-- help with detecting those events.

ebizQ: So when should company should adopt complex event processing, and when should a company adopt business activity monitoring?

Gualtieri:
Let me answer that, but let me take a step back on the complex event processing.

Complex event processing is a relatively new and very complicated technology because it does temporal pattern recognition at very low latencies. That sounds very complicated, but basically, what it means is it can detect if two things happen very, very close together.

The initial market for CEP engines was none other than Wall Street because you’ve got an enormous flow of information. So imagine Microsoft’s [stock] price goes in at $27.50 and then IBM goes in at $90 within 3 milliseconds. That's something that CEP can detect and, of course, then you might want to take some action on that. So that is an extreme case [but] that’s CEP's sweet spot and there's very, very low latency transactions.

In business activity monitoring, you may have--and you often do have--similar circumstances. It may not be in the millisecond level, but you'll have a sequence of events that alone don't tell you anything. But together, it's a pattern that you then want to take some action on to avoid a missed delivery or keep a manufacturing line running, for example.

ebizQ: What are some the internal barriers to adopting either technology?

Gualtieri:
It’s always the problem with adopting any new technologies that [doing so] often requires connections to a lot of data sources. When you have very large organizations that have a portfolio of 300 applications and a very heterogeneous infrastructure, just connecting to and capturing those data sources is a key challenge--and the first challenge that you have to deal with in adopting these technologies.

The second challenge is to really identify what patterns actually matter, because there's a lot of noise. You can imagine all the data running through; some of it is not relevant, some of it’s noise. You have to find the signal in that data. So some analysis work has to be done by a combination of business and technical people to figure out what matters, [answering questions such as] “What do we actually want to detect? What are those patterns we want to detect?” That can be a lot of work.

But the good thing about business activity monitoring is you can start small. You can just focus on “if we could just know this one thing, then we could increase our on-time delivery.” You can focus on one end so there can be a payback from focusing on one narrow business outcome.

ebizQ: That makes sense. Now can you give me some real-world examples of how complex event processing and business activity monitoring can actually benefit a company?

Gualtieri:
Sure. The sweet spot of complex event processing is very, very low latency temporal pattern detection. “Temporal” just means “in time,” detecting patterns in time. I already mentioned the Wall Street example, algorithmic trading. [CEP] is used in all of those trading houses for algorithmic trading.

But the opposite of that is fraud detection. Regulatory authorities are using complex event processing to detect fraud. If this trade happens and this money moves to this bank account, hmm, we might have some fraud, because there's a sequence of events there that may indicate fraud. So you can imagine that same approach can be used on web traffic and for security purposes. So those are two obvious examples.

Some less obvious examples are in transportation and logistics, which is an extraordinarily event-driven business, or the airline industry, where if a certain event happens, it can cascade and cause problems. [For instance,] a missed flight can cause missed baggage.

There's a whole cascade of events that can happen in a business, in any business that is very event-heavy. Think of manufacturing, where the consequences of something going wrong can have a domino effect. Then CEP is a good fit.

ebizQ: What are some of the key best practices for companies adopting complex event processing?

Gualtieri:
They need to be able to find models. Let me explain that. Complex event processing is an execution platform. So you give it a pattern and you say “Find this pattern when it happens in large streams of data, high-velocity data. Given this pattern, I want you to find it.” That's what CEP does.

But what CEP doesn't do is it doesn't tell you what pattern you need to find. So you need analytic work. It might be some predictive analytics or advanced analytics, or even business rules to figure out what those patterns are that matter. That’s kind of the first step.

So many people think that CEP does real-time analytics. No, it's an execution engine. You can create a model that will help you create real-time dashboards and detect systems, but you have to find the model offline.

ebizQ: What would you say are some of the key best practices, then, for business activity monitoring?

Gualtieri:
The first one is to find all the business outcomes that you want to improve. It’s great to start with [key performance indicators (KPIs)] because that's the low= hanging fruit. So you make a list of all those key performance indicators.

Then you're going to have to go up and down, looking at the KPIs, looking at the data. [You need to] create a shorter list of initial KPIs because you don't want this big-bang project where you say “All right, we want business activity monitoring for all of these KPIs.” That may be the goal, but your implementation could be very, very complicated. It could take a long time; you could struggle to find the data. So you want to prioritize it and phase it in so you get the value from it as quickly as possible.

The second best practice, from a technical standpoint, is what I mentioned before: data integration. You have to find the connectors. There are products out there now that are able to deal with an enormous amount of volume that prior technologies weren't [able to do so]. You may have heard of companies like Splunk, or Sumo Logic, or Tibco LogLogic, which actually collect enormous amounts of information from IT systems and then compress, store, analyze and create dashboards from that.

ebizQ: This sounds like fairly a revolutionary technology, which means mistakes are crucial to avoid. So what are some of the biggest mistakes, and how do you avoid them in either of these technologies?

Gualtieri:
Well, a lot of the vendors--not just the ones I mentioned, but CEP vendors, too--they do tend to sort of over-sell and over-simplify. There’s that point I made that you have to find the model that you want to detect for CEP. You just can't just hook CEP and say, “I'm going to put all this network traffic in here; I'm just going to have it detect fraud.” No, you have to have an expert that knows what you're looking for.

So that's a key thing to understand, that's the key step back that you need to take before you adopt these technologies.

Other than that, you also have to look at how you are going to get this information to the people or other systems that need it. So fine, you've monitored something, you've detected something of interest. What are you going do about it? Can you do anything about it?

You might look for systems that can hook this up to existing application systems, expose it as a Web service, create dashboards for people to monitor, [or create] alerting systems so you have to look at the output of these systems and how you can actually connect it to the actions that you want to take in the business.



About the Author

Peter Schooff is a former contributing editor for ebizQ, where he also managed the ebizQ Forum for several years. Previously, Peter managed the database operations for a major cigar company, served as writer/editor of an early Internet entertainment site and developed a computer accounting system for several retail stores. Peter can be reached at pschooff@techtarget.com.

More by Peter Schooff

About ebizQ

ebizQ is the insider’s guide to next-generation business process management. We offer a growing collection of independent editorial articles on BPM trends, issues, challenges and solutions, all targeted to business and IT BPM professionals.

We cover BPM standards, governance, technology and continuous process improvement, as well as process discovery, modeling, simulation and optimization, among many other areas. We follow case management, decision management, business rules management, operational intelligence, complex event processing and other related topics. We closely track important trends such as the rise of social BPM, mobile BPM and BPM in the cloud. We also explore BPM’s use in functional areas, such as supply chain and customer management, and in key verticals, such as financial services, health care, insurance and government.

ebizQ's other BPM-oriented content includes podcasts, webcasts, webinars, white papers, a variety of expert blogs, a lively online forum and much more.

-1-

1  

Explore Our Topics

  • EDITOR'S BRIEFING
ebizQ editorial highlights and updates, compiled by ebizQ staff
  • BaaS bonus: In this podcast, Gartner Research Director Gordon Van Huizen discusses the Backend as a Service marketplace and the approach's pros, cons and challenges.
  • Cloud quiz: How much do you know about business process management in the cloud? Test yourself with this six-question cloud BPM quiz.
  • Three key steps: Clay Richardson of Forrester Research describes ways to emphasize user experience in business process design.
  • ebizQ editorial: Browse ebizQ's collection of independent editorial content, including articles, tips, Q & As, podcasts, guest columns, book excerpts and more.
  • Virtual Conferences
  • Webinars
  • Roundtables

BPM in Action

March 10, 2011

The sixth annual BPM in Action 2011 Virtual Conference will explore cutting-edge market developments in BPM and describe how to leverage them for improved business operation and performance. More

View All Virtual Conferences

Smart Case Management: Why It's So Smart.

Date:Nov 05, 2009
Time:12:00 PM ET- (17:00 GMT)

REGISTER TODAY!

Date:Oct 29, 2009
Time:15:00 PM ET- (19:00 GMT)

REGISTER TODAY!
View All Roundtables
  • Research Library
  • Podcasts
  • News

Joe McKendrick: Part II of II: Designing Evolve-ability into SOA and IT Systems

In part two of Joe McKendrick's recent podcast with Miko Matsumura, chief strategist for Software AG, they talk about how SOA and IT systems need to change and grow and adapt with the organization around it.

Listen Now

Phil Wainewright: Helping Brands Engage with Social Media

Phil Wainewright interviews David Vap, VP of products at RightNow Technologies, and finds out how sharing best practices can help businesses understand how best to engage with online communities.

Listen Now

Peter Schooff: Making Every IT Dollar Result in a Desired Business Outcome: Scott Hebner of IBM Rati

Scott Hebner, Vice President of Marketing and Strategy for IBM Rational, discusses a topic on the top of every company's mind today: getting the most from IT investments.

Listen Now

Jessica Ann Mola: Where Will BI Fit In? Lyndsay Wise Explains

In BI, this tough economy and the increasing role of Web 2.0 and MDM are certainly topics on people's minds today. WiseAnalytics' Lyndsay Wise addresses each of them in this informative podcast.

Listen Now

Dennis Byron: Talking with...Deepak Singh of BPM Provider Adeptia

Deepak Singh, President and CTO of Adeptia, joins ebizQ's Dennis Byron in a podcast that gets its hand around the trend of industry-specific BPM.

Listen Now
More Podcasts
  • Most Popular
  • Quick Guide
  • Most Discussed

Quick Guide: What is BPM?

Learn More

Quick Guide: What is Event Processing?

Smart event processing can help your company run smarter and faster. This comprehensive guide helps you research the basics of complex event processing (CEP) and learn how to get started on the right foot with your CEP project using EDA, RFID, SOA, SCADA and other relevant technologies. Learn More

Quick Guide: What is Enterprise 2.0?

A lot of people are talking about Enterprise 2.0 as being the business application of Web 2.0 technology. However, there's still some debate on exactly what this technology entails, how it applies to today's business models, and which components bring true value. Some use the term Enterprise 2.0 exclusively to describe the use of social networking technologies in the enterprise, while others use it to describe a web economy platform, or the technological framework behind such a platform. Still others say that Enterprise 2.0 is all of these things. Learn More

Featured Bloggers

Steven Minsky's Latest Blog Posts:

Read Steven Minsky's Blog
Scott Cleveland's Latest Blog Posts:

Read Scott Cleveland's Blog
Adrian Grigoriu's Latest Blog Posts:

Read Adrian Grigoriu's Blog
Noam Tamarkin's Latest Blog Posts:

Read Noam Tamarkin's Blog
Janne J. Korhonen's Latest Blog Posts:

Read Janne J. Korhonen's Blog
Dion Hinchcliffe's Latest Blog Posts:

Read Dion Hinchcliffe's Blog
Michael Poulin's Latest Blog Posts:

Read Michael Poulin's Blog

View All ebizQ Bloggers