We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.
2 vendor_downloads items found, showing 1-2 Sort by Relevance / Date
08/13/2007 Full Transcript: Event Processing: Competitive Advantage Through Situational Awareness Gold Club Protected
Event Processing: Competitive Advantage Through Situational Awareness Beth Gold-Bernstein BGB:    Welcome everyone and thanks for joining us today. I’m Beth Gold-Bernstein, Director of the ebizQ Learning Center and moderator of today’s program. Before introducing the webinar, I’d first like to take a moment to acquaint with the features and functions of the environment which is highly interactive. You can chat with company representatives and other visitors, send a business card or leave a message. After today’s presentation, we will be taking your live questions. You can submit your questions by clicking on the “Ask a Question” button. To download a copy of today’s presentation, click the “Files” button. To enlarge any of the slides, click on the magnifying glass icons to the right of the slide.

Today, we are very happy to welcome back Roy Schulte, Vice President and Research Fellow at Gartner who is going to be talking about Event Processing and Competitive Advantage through Situational Awareness. This webinar is a preview of Gartner’s upcoming Event Processing Summit which will be held September 19th-21st in Orlando. I’d also like to thank BEA for sponsoring this educational event. Now, I’d like to turn the program to Roy. Thanks for joining us, Roy.

Roy SchulteRS:    Thanks, Beth. Thanks for inviting me. It’s always a pleasure to work with you on these webinars. The topic today of event processing is something that is very important. It’s becoming of greater interest to Gartner clients and, in fact, to the industry as a whole. The presentation today is an introduction to this field of event processing, to provide backgrounds to people who are going to attend the conference in September or just for anyone who is interested in the topic and wants to learn more about it.

Take our EDA Poll and Win an iPod

ebizQ is conducting a survey on event-driven architectures (EDA). Event-driven processing can increase business agility by providing faster, deeper insight, and responsiveness to business-impacting events as they happen. This 15-question survey only takes minutes to complete. Participants in the survey will be entered into a drawing to win a fabulous new Apple iPhone.

Our Upcoming ebizQ SOA Webinars

1. SOA in Action: Another Virtual Show
Review our Agenda for our Second SOA in Action Virtual Conference on Oct. 30 and 31st.

This presentation will provide an explanation of why architects and software engineers, business analysts and IT managers are using event drive architecture more often in business applications. So, in today’s session we’re talking about how you get competitive advantage through situational awareness. Now, situational awareness, sense-and-respond and track-and-trace are all very important issues for companies today. They are management strategies. What makes it interesting is that they have some very important things in common. What they have in common first is that they are all aspirations – things that people want to have and they are all associated with certain themes such as:

•    Adaptive enterprise
•    Agile enterprise
•    Zero-latency enterprise
•    Real-time enterprise
•    Prediction-based action

Now, situational awareness implies having an up-to-the-minute understanding of all of the critical aspects of the environment and your own internal operations. So, simply, it’s just knowing what’s going on so you can figure out what to do next.

Sense-and-respond has a similar flavor to it. In fact, I see people using these terms synonymously with situational awareness. However, sense-and-respond emphasizes the early detection of specified opportunities and threats perhaps using only one or a few information sources, whereas situational awareness connotes seeing the big picture – a holistic view of all of the factors that are giving you input from many different places.

Track-and-trace is coming at this from a slightly different angle. It’s talking about the recording of tasks and the presence of identified items as they move between physical locations or through logical steps in a business process. For example, logistics systems track and trace the movement of physical goods in the supply chain. Supply chain management systems, then tell people where the shipment was, where it is now, they can predict when it is going to arrive at the destination and they can prove that a shipment was delivered. The idea of track-and-trace is applied to supply chain, it can also be applied to any straight through process like moving insurance claims or customer order or payments or other types of intangible information-based items as they go through their life cycles.

Now all three of these concepts are examples of operational business intelligence. Each is intended to decrease the gap in time between something is known or is knowable or at least is predictable and from a person or device or an organizational unit, take action on it. Situational awareness, sense-and-respond, track-and-trace are modes of operation and event processing is how those modes of operation, how those aspirations can be achieved.

Event processing is a set of specific concepts, design patterns, best practices and software tools and that’s what we’ll talk about as we go through this presentation. So, let’s begin this topic by looking at some of the terms that we’re going to use.

Event Processing Terminology

First, an event. An event is exactly what you think it is. An event is something that happens. Could be a bank transaction, a stock trade, could be a customer order, could be someone changing their address, could be the delivery of a shipment or could be a major event like buying a house. Now, of course, computers can’t deal with abstractions like that. Computers have to deal with bits and bytes, 1s and 0s. So, to make an event computable, we make event objects.

Some application system or some device has to create a machine readable report of the event, called an Event Object which is associated with that real world event. For example, if Fred Smith deposits $100 in his bank account at 10 today, that’s an event. The computer record associated with that deposit transaction, perhaps in the form of an XML message is the event object. Event processing means computing that performs operations on these event objects like creating them, reading them, transforming or deleting them.

An event object may be in the form of an XML message, it could be in the form of some other kind of message, it could be an email or it could just be a row in a database. Anything that gives a specification, a record of an event is an event object. An event processor is a software module that sends or receives or somehow manipulates an event object.

Are Companies Event Driven?

Now, when you talk about event processing, one of the first questions most people ask is “Aren’t companies already event driven?” after all, events just mean that something happened. And the answer is, of course, “Yes”. Companies have been event driven ever since they’ve existed. Long before we had computers, companies were event driven. Thing happened, people bought things, people sold things, people did transactions. However, many of the application systems and many of the automated processes that we use in business today are event driven only at the surface level. Deep down, if you look at the design of these systems, they are not event driven in their underlying application architecture.

In the example, here we have an order entry application. You have a person at a counter or perhaps a person over the internet placing an order for some goods. The most common way of implementing this would be using an interactive, a request/reply kind of design pattern where a request comes in, some data is captured and then some back end processing is done. Perhaps people look up a price, they look up the availability of the goods, they look up someone’s credit rating and so forth, and the order is captured in a request/reply fashion. Many of the things are also done in a request/reply or “pull” based fashion. When you look up your bank balance, that is a request/reply.

So, whether you’re doing it in person or whether you’re doing it on the internet, a great many applications are doing this, including the majority of service oriented architecture applications that are in use today. Today’s SOA applications are request/reply between the user, the end consumer and the applications system and then within the application system, ach of the services that are invoked are often invoked in a request/reply fashion.

If it is a complicated process, a multi-step process, the next step in the process even in a world of SOA is often done on a pre-scheduled basis or “batch” basis. So, you might capture the order in a request/reply fashion, put it into a database and then at night, a timer goes off and the company’s internal clock starts up an order fulfillment application or a manufacturing application to take that order out of the database and then process it in its next step. These two things – request/reply which is a pull or a scheduled operation are not event driven operations.

However, we could turn this into an event driven process under the covers by redesigning how it’s implemented. We might still capture the order using an interactive request/reply order entry system, however, once you’ve captured, rather than put it in a database and have a batched out fire up, instead we might dispatch a real time message or near real time message which essentially is containing the event object, the record that someone has placed an order and that acts as a transfer, a trigger, for the next step in the process which could be manufacturing or order fulfillment or perhaps billing and so forth down the chain. In this case, what we’ve done is we implement a multi-step business process as a staged event driven process. Each stage in the process is triggered by the arrival of an event object.

A second way in which we use an event driven architecture in this particular example is by using an event delivery to notify somebody that something has happened. So, for example, when the article is finally shipped, we might want to send a notification message, an email or some other sort of message to the person saying that the goods have shipped. Again, what’s happened here is that we’re using an event, a push of an event object to notify a person that this has happened.

In this way, we’ve done several things that help the process run faster. First, we made it run in near real time end to end, we’ve made the stages so that they don’t have to wait for end of day or some other batch cycle take place. We’ve made it end to end. Second, we’ve also made it so that the person doesn’t have to poll continuously and keep looking when something has changed. You’ll notice more and more of the activities that you’re doing with business are already done on this basis, are already done on a push basis. For example, if you’re credit card company notices that some strange activity is happening on your credit card, you will be notified proactively. They will send you a message or they may call you up, saying “we’ve noticed some strange activity on your credit card, maybe we need to check into that”. So, to speed up the functions of a company or of a person, this mode of being eager, this push mode of behavior is being used on a wider basis.

Now, this is not a new idea, of course. Some people have been doing this for many years. This has been called message driven processing in some circumstances. In many places, it’s also called “stage two processing” or “flow-through processing”. Nevertheless, what we’re talking about here is a simple use of events. Now, we call this simple in the sense that the arrival of one event is sufficient to trigger the operation of the next step. And that becomes important when we look at the other types of event processing coming up.

To summarize, an event driven architecture is different than the way many times we develop a typical business application. First, because we’re using events that were pushed from one step to the next step. Second, the recipient of the event act immediately, not on a timer basis and not when a request is made from some other place. And third, one of the attributes of an event object is that the source of the event, the sender, doesn’t specify what action the recipient of the event is going to perform. When someone sends an event, all they’re doing is tell you that something has happened or that something is predicted to happen. The choice of what to do about it is made by the recipient. What this does is de-couple the sender from the receiver in a way that is less, so that the sender and receiver are less bound together than they would be in a traditional request/reply relationship.

Request/Reply Relationships and Event Processing

In a request/reply relationship, the requester specifies what the receiver is supposed to do; you’re supposed to place this order or you’re supposed to look up a credit history or you’re supposed to do something. In an event driven architecture, the event object narrowly reports that something happened and the event receiver, the consumer of the event decides what to do. This gives the developer flexibility to change the receiver, change the consumer without having to change the sender. It also gives you the ability to add additional receivers to the same message so the same message can be delivered to multiple places. Again, without changing the sender. This gives you a flexibility, a plug-ability beyond what is found in a typical request/reply kind of situation.

What we’ve discussed here are the potential pulls of a simple kind of event processing, the one at a time kind of processing. But this is really only half the story of event processing. The second aspect of exploiting event processing is to look at the information value that is available in multiple events and this is called “complex event processing”.

In this kind of a system, the event processor is looking at multiple events, often of different types, coming from different sources. The event processing system may look at the relationships between the events to find patterns, to connect the dots, to extract insights that are going to enable faster and better operational decisions. In some cases, the purpose of the system is to drive a dashboard that a person is looking at, a BAM dashboard – business activity monitoring dashboard. In other cases, the event processing system is sending an event to a business process manager or an application program to kick off a business process that’s going to operate in response to the information that’s been found by the event processing agent. So, here, what we’re looking for is threats and opportunities. We’re searching through the date using it’s information value to find opportunities to get competitive advantage or places where we can detect risk and things that we want to avoid.

A classic example of this is found on Wall Street. They’ve been doing complex event processing, in the form of events being processing for 15-20 years. In any algorithmic trading application, what you do is you look at what’s happening in the market, you want situational awareness for what’s taking place in the capital markets. In this case, the event source is the stock market exchange. They are the ones who are recording the trades and they’re also putting out price quote information. The transfer of the data comes from the stock exchanges, sometimes through a data, real time data provider like Reuters, Bloomberg and the event object are stock picks, the actual data of the trades or the price quotes. The event consumer is an event processor and it’s built into a trading platform. The trading platforms are rules engines that are capable of doing algorithmic trading; they’re applying rules to decide when to do a certain trade. What they are doing are looking for patterns in those events, they’re calculating average prices perhaps in the last one minute or five minutes, they’re looking at trends of price over time, they’re looking at the spread between the prices in one location and in another location and so forth. Based on the findings, the insights they can derive by looking at these event streams, they then may take an action immediately to buy or sell.

Now, this is a classic kind of event processing application, a complex event processing application where you’re dealing with a very high volume of data, sometimes of thousands, sometimes tens of thousands of messages per second. And you have to respond very quickly because the opportunities come and go within milliseconds. An algorithmic trading engine may be able to respond in a few milliseconds where a person couldn’t possibly respond that fast. So this is a place where you have really an extreme in terms of a many things both the volume and the speed at which you have to react. The actually complexity in the computation is not all that great.

Complex Event Processing

So, what we’ve done is we’ve illustrated a second kind of event processing, here. Here, what I was just talking about making an end to end business process that runs smoother and faster and more straight through, here, we’re talking about using complex event processing for purposes of operational business intelligence. The properties of this kind of event processing are a super set of the simple event processing we talked about earlier in this discussion.

First, these systems are event driven, they use an event driven architecture so again the events are pushed, the recipients act immediately and the event source doesn’t specify what the event recipient is supposed to do about it. However, beyond that, there are these additional capabilities that are added to the picture. First, they are looking at multiple events at one time. An event stream processing system is sort of a simpler, but sometimes very high volume and very fast moving type of complex event processing. Typically, you’re dealing with only one or two event stream and typically, what the system has to do is to filtering very quickly. By filtering we mean separating the wheat from the chaff, throwing away the events that are not of interest, to be able to pick out the events that do matter. From this it does calculations: adding things up, averaging things, doing some simple operations or comparisons to find out what action should be taken.

Beyond this there are also other kinds of more sophisticated types of complex event processing. They may be looking at things like causality: why did this event occur, what events happened previously that led to this event taking place. So this is sort of like watching the dominoes fall and understanding that when this domino fell, there were a chain of dominoes that fell before it. Being able to trace back in time what the other events were that led to this particular event.

Talk about event hierarchies where you consolidate the input from a number of events into one complex event and then you may further consolidate the information value in multiple complex events to yet a higher level complex event. The idea is to distill down many simple readings into a more summary level meaning, summary level event which actually is easier for a person or an application system to assimilate. What they want is the bottom line, not the details.

If I give you a complex event that says, “gee, your order is about 20% between 8 and 10 this morning”, I’ve told you something that you can act on. Whereas if I give you a detailed listing of the 350 orders that came in this morning, you wouldn’t be able to make much value out of that, those details. So, in fact, complex events are called complex but what they actually are is something that is simpler to assimilate because they are a summary of the information value and other events.

Complex event processing is one of the fundamental underpinnings in many kinds of business activity monitoring. When we talk about business activity monitoring, many times we’re thinking of dashboard systems. Systems in which someone is looking at a browser-based display of some key performance indicators. The display may include bar charts, it may include pie charts or it may include other representations. It doesn’t have to be graphical, but what you’re trying to do is track in real time or near real time what is happening in your company or in some part of your company. There are many different kinds of business activity monitoring. It doesn’t have to be done on a dashboard. You can be doing business activity monitoring and have the access channel be an email, or it could be a phone message or it could be some other channel, some other way in which the computer is helping to notify what happens with a person.

Also, many times, people would consider business activity monitoring part of triggering an automated process. So, in some cases, the response to something that you’ve detected through the system is not just displaying it to a person but it may be starting up an automated process to respond to that thing. Some of the systems that do business activity monitoring have sophisticated alert management capabilities. What they will do is to use a role based mechanism to alert a particular person or a particular type of job category when something has happened – an exception condition, a threat or opportunity has been detected. If the person who has been notified doesn’t respond back, then the alert management system may escalate and send the alert to someone else or try the same person on a different channel. So there are many ways in which the concept of the business activity monitoring can be implemented but in every case, what you’re doing is you’re using business events behind the scenes to help detect what the situation that you want to operate on.

Now, and sometimes the business activity monitoring tool itself has within it an event processing capability; in other cases it is using a separate external complex event processing engine. Now, business activity monitoring can be considered to be the real time operational side of a holistic business intelligence strategy. Now classic business intelligence of course, looks different than this. In a classic business intelligence effort, the end user is a manager or an analyst making some sort of strategic decision. On the other hand, by contract with a business activity monitoring system you’re generally talking about an operational person making a tactical decision on a moment by moment basis. Traditional business intelligence is like looking for a needle in a haystack. The haystack is the data mark or the data warehouse and the business intelligence tool is able to look through vast quantities of data and find the important facts that help you make those strategic decisions.

Business activity monitoring based on complex event processing works a little differently. It is sort of more like finding a needle in a conveyer belt of hay. The conveyer belt of hay are the facts coming at you in a real time stream. The facts are the event objects. And the needle you are looking for are the things that are changing, the threats and opportunities that you can detect in near real time. Now one of the big aspects of business activity monitoring, just like the rest of other types of more traditional business intelligence is to separate the wheat from the chaff, to separate the important facts from the less important facts. In business activity monitoring there’s a premium on getting the results out to the person quickly because, in many cases, you’ll have a relatively short period of time to respond. In the case of trading systems, you have milliseconds to respond and so you have to do it on an automated basis. In other kinds of business process monitoring, you may have seconds, minutes or even hours to respond depending on what type of business process it is you’re monitoring.

Reasons for Event Stream Growth

The number of places in which companies have event streams is growing rapidly and this is happening for several reasons. First, we have the ever-decreasing costs of hardware so the costs of sensors are going down and the cost of the computer, the servers are also going down. Sensors, things like RFID and GPS and bar code readers and so forth have become less expensive. Because of that, we’re able to buy more of them, we’re able to instrument many more activities than in the past we would have been able to. We also have, of course, much faster networks, great bandwidths and decreasing costs for sending data over those networks and that’s very important for enabling event processing applications.

 Last, but not least, we have the web and we have Web 2.0 now and other aspects of the internet which are increasing the access of consumers and people working in business to get at data and to directly input and deal with the computing applications. The standards that are associated with the web have made it possible to build applications more quickly and have those applications talk to each more easily. Because of this, we’re seeing brand new kinds of event streams that we’ve never seen in the past. The volume of events and the number of different kinds of event streams is growing rapidly in almost every company. We have many kinds of event streams. Some of them old and some of them new. The traditional transaction processing systems like bank ATMs and airline reservations and insurance claims and so forth, those are growing in volume just because of the nature of businesses expanding. The sensor networks really are new kinds of things, something that most companies didn’t have 10-15 years ago but many companies are developing them based on the new types of devices that we just talked about a minute ago.

The web has been the source of many new kinds of event streams. We have web auctions, we have quickstreams and quickstream analysis which is a very fast growing example of an event processing application. We have RS and atom feeds (??? 27:53.9) which are push based mechanisms or at least they appear to be push based mechanisms for telling people when something has happened, when events have happened. We also have transaction streams that are consisting of very high volume sort of different than traditional LLPP in a sense, they really are a transaction processing still. In telecode, we have call data records, we have credit card transactions both for the authorizations and for the actual buying of things and we have micro-payment systems now where you have a vast number of relatively low value things. The net of all this is that companies are being flooded with event streams today. A large enterprise today has any where from 10,000 to 10 million business events per second taking place. And if you multiply that out that means that there are literally trillions of business events happening in companies today.

The goal of event processing is to harness these events, to harness the information value of these events to help companies to act better, to help the companies make better decisions, smarter decisions and faster decisions.

Harnessing Events for Event Processing

When we look around, we see a number of applications that are being used that leverage this today. We talked about algorithmic trading which is one of the classic and one of the earliest applications on Wall Street where event processing was being used. Now, they could afford to do it 15-20 years ago because of the value of what they were doing was so high. Now, because of the cost of event processing coming down and the spread of the event processing technology, we’re able to use it in many more applications and mainstream companies covering a lot of different kinds of activities.

One of the most widely used types of event processing systems is for compliance reporting and monitoring. You’re keeping track of what’s happening for regulatory purposes. Related to that is fraud detection, things like anti-money laundering, credit card usage and telco – phone card – usage also. The military has been using event processing for years and it’s a very important part of battlefield operations today, sometimes using satellites and other ways in which they’re using military intelligence. The military played a key role in the development of event processing technology because they funded a lot of the research in the late 1990’s and early 2000’s where the academic institutions were inventing the algorithms and the techniques that are used for event processing today in other industries. In fact at the Gartner Conference in September, one of our case studies is on a military intelligence application from the U.S. Army.

The pharmaceutical industry is using track and trace. Using event processing to keep track of the location of shipments of pharmaceuticals from place to place. The goal here is to track the pedigree of each individual unit, to know that you haven’t had a counterfeit drug swapped in because you were able to keep track of where the actual legitimate drugs are at every moment in time. Security is one of the biggest uses of event processing today, things about intrusion detection and this is both intrusion for a physical places like in an art gallery, being able to detect when someone has entered the art gallery and may be stealing something or a network intrusion where someone has penetrated a network and is perhaps going to cause havoc on a network basis.

Transportation operations are another place where event processing is being used already. The year one (??? 31:49.8) operations, the chain operations, shipping companies and so forth, keep track of what’s happening on a day-to-day basis, on a minute-by-minute basis. This is an example of situational awareness where there are multiple different event streams all contributing to an overall holistic picture of what’s happening in the enterprise.

Event Processing and Service-Oriented Architectures

Now, at this point some people may be saying “sounds interesting but what does all this have to do with our other main strategy today?” and that’s the service oriented architecture. And, in fact, there’s a very high overlap between event driven architecture and service oriented architecture. When you use an event driven architecture, you have a choice of how to do it. You can either implement the event driven architecture as a service oriented architecture or you can implement it without using service oriented architecture principles. Most of the time, if you’re dealing with business events today, you should be using service oriented architecture principles as you build an event driven application.

Now, for something to be called a service oriented architecture application, what we mean is that it conforms to the five principles of SOA. First, the application must be modular, it’s of multiple pieces. In this case it would be event senders, event sources and event consumers would be the modules. The modules have to be distributable across multiple computers. There has to be a formal interface, a definition of what is happening between the event consumer and the event source. Now, the interface for an event driven system is a little different than the interface for an expressly applied SOA system but in both cases, some things are the same, that is in both cases the contents of the messages that are sent back and forth have to be documented as the interface. The difference is that in an event driven architecture, you don’t have the notion of specifying what the operation is whereas in a request/reply kind of SOA, you do specify an operational method name.

The fourth principle is you separate the interface from the implementation meaning that you can create a new source or a new event consumer using the same definition, the same description of what the message is without having to change anything else in the system. And finally, the modules are sharable so a consumer can accept events, event topics, multiple sources and multiple sources can send the same event object to multiple consumers. So, if you implement an event driven architecture application using those five principles, then what you have is a SOA application that is event driven.

When you’re doing this, you can implement the event driven architecture application with a lot of the same techniques and people and tools that you use for the other kinds of SOA applications. You can leverage standards like XML schema description XSDs. You can send the events over a SOAP message and you can use WSDL(???) to define how the communication is done for EDA. When you’re doing event driven architecture this way it should be coordinated by your service oriented architecture center of excellence, assuming that you have one. That is, you should not have two different groups in your company, one to do event driven architecture and one to do service oriented architecture. Rather, you should look at event driven architecture as part of your SOA strategy. You can also share the same middleware, the same backbone, so the same enterprise service bus or the same message oriented middleware system or the same web services stacks that you’re using for your classical request/reply kind of SOA can also be used for sending the event driven architecture messages from the source to the destination. So, what happens here is that you have event driven architecture parts of your application co-existing with your request/reply parts of the same application system.

Now, having said that, there are all those similarities between event driven architecture and the service oriented architecture but there are some differences between EDA and conventional SOA systems. As we described earlier, event driven architecture means you’re dealing with event objects, you’re dealing with notifications that something has changed whereas a request in a SOA system that’s request/reply doesn’t look the same. In a event driven architecture you’re also using the event objects, not just to run an application or a composite application as you would in a request/reply, but you’re combining the information that’s in multiple event objects to derive a higher level insight, you’re connecting the dots together which is, again, something you wouldn’t do in a typical service oriented architecture application. What this means is that the architects and developers, when they’re building SOA applications using EDA, they have to be aware of the difference in some of the design concepts while still, at the same time, using the same tools and many of the other aspects of classic service oriented architecture to build the system.

Now, event driven architecture is something that is expanding very quickly. If you look back through your four years, you find very few complex event processing capable tools on the marketplace. At that point, there were some interesting projects that had been done in academia but there were very few commercial products on the market that did complex event processing. The simple kinds of event processing, of course, were always available, that is, you could do a staged event driven process using single events for decades, however, off-the-shelf products for the complex event processing are something that is fairly new to market. At this time, Gartner is tracking more than 30 companies that are offering business activity monitoring and complex event processing software. Some of these are small companies, started out just to do this. In other cases, we’re talking about major companies, large software vendors who have implement event processing and business activity monitoring as a part of their offerings. This area has actually been marketed less by the major vendors than perhaps one would have expected by now. Many of these companies have invested a lot in terms of developing the product but they haven’t talked that much about them in public and spent a lot of money marketing that. We think that this will change over the next several years as the market becomes more aware of event processing and the value you can derive by exploiting it.

Recommendations for Event Processing

So, let me conclude with some recommendations here. First, if you’re doing business activity monitoring or other similar projects that require real time insight into what’s happening, things like situational awareness, sense and respond (???39:15) or track and trace, you should be looking at how to apply complex event processing to the situation to be able to achieve your competitive advantage. Today, most of the applications that do this would be considered leading edge or at least early adopter’s in their category and you’d probably have the advantage over your competitors.

However, in the future we’ll find that you’ll have to do this just to keep up because if you’re competitors are using it and you’re not, you will be at a competitive disadvantage. We believe that you should make event driven architecture part of your SOA strategy from the beginning. Don’t implement SOA only with request/reply in mind. Implement SOA using the choice of event driven architecture or request/reply from day 1 so that depending on what the particular business problem you’re trying to solve looks like, you’re using the right tool at the right time. Your architects and your developers should understand event processing; there will be a learning curve for most people because in many cases it requires a different way of thinking than what they traditionally do. By default, most application systems uses a pull, request/reply or a batch scheduled kind of thinking so what we need to do is make sure the people are aware of the option here to use event driven architecture when it’s appropriate.

The Gartner conference on this topic coming up in September is going to cover this in a way that’s a little bit unique for us. At Gartner, we’ll be having some Gartner analysts to describe the trends in the market and to give advice in a vendor-neutral fashion. In addition to that, we have 13 case study speakers so we have a lot more case studies than we would have at a typical conference talking about the use of event processing. One of our tracks is specifically dedicated to the finance industry because that’s a place where there’s a lot of activity going on today. The other two tracks are tracking other uses of event processing and thing like security and supply chain and military applications, electricity trading and so forth. We’re also delighted to have two of the developers in this field, two of the innovators from academia who invented much of what’s happening here to deliver speeches at this conference. And finally, we have 16 vendors who offer products in this space and they’re going to demonstrate their products side-by-side and so you can take a look at what’s happening in the marketplace and what is the state of the art for the commercial software tools.

So, with that, let me turn the microphone back over to Beth.

BGB:    Thank you very much, Roy and in a few moments we’re going to go to your questions. I’d like to remind you, you can ask a question by pressing the “Ask a Question” button. But first, we have a few words from our sponsor, Guy Churchward, Vice President and General Manager of WebLogic Products, for BEA Systems is going to say a few words. Welcome, Guy.

GC:    Thank you, Beth and thanks very much to Roy for the presentation; it was excellent. This is Guy Churchward. I’m Vice President of WebLogic products for BEA and wanted to talk about, just briefly, around WebLogic event server and WebLogic real time, two products we launched very recently and they’re our entry into EDA, event driven architecture. Our aspirations as we show on this slide are really around instantly responsive enterprise and what I would like to stress is what BEA is about and WebLogic is about as rock solid IT infrastructure for the enterprise and again, these features whether they’re event server or real time are really driven out of our need from our customer base, our clients have really been asking for this as being the next, sort of iteration in their experience of moving forward.

Our key takeaways we just got from Roy’s slide obviously from the demand side been talking about how this is really exponentially growing the events so whether it’s going to be a 10th power four, 10th power 7 on the business events per second and there’s virtual enterprise.

So, basically, where the events are coming from, whether it’s the central network or whether it’s data feeds, whether it’s coming into BPM, it is just an explosive industry and the challenge is that the current situation is the applications invariable are written in C, C++ and people really want more of a commodity and a faster way to be able to adjust to it whether its agility or a competitive advantage and that’s where we come in on this job assignment. We wanted to bring really the values and the agilities around the events server into the world of Java and then into our client base. In the demand for it and market space is definitely on our beta cusp, we’ve seen high demand from financial services and telcos, manufacturing, logistics and transportation so it’s been pretty interesting to see, the sort of evolution of this, I mean, originally, we’ve moved into the market around the financial services but there’s been a huge thrust even in things like commodity control for military and then systems management which we never really saw coming but it’s been an explosive growth on it so we’ve been really excited.

And then, obviously on the supply side, what people are really looking to do is to be able to move into harnessing the events that are coming to their businesses as quickly as possible but in a less expensive way and now through, you know, hardware advancement and things like in-memory cache and innovations around complex event processing then it initially enables people to move towards this business in a lot more competitive way. But you do need the ultimate agility so when you are changing roles, when you are changing dynamic approaches to get competitive advantage, you want a very quick way to get to market and that’s where we see the Java and running it on a platform such as BEA’s offerings is a good way to do.

So, what we’ve brought to market is the first complete infrastructure for event processing. It really means that there’s an application server as an event processing engine so you actually have inside of it the complete life cycle, whether it’s development to production but the concept is that it should be able to run and manage in the same container both the artifacts that you have from development but also the platform we things like an event streaming engine, complex event processing and then running this with including the same JVM for extremely high performance of the application usage. We’ve tailored the application server specifically for the event processing market so this is again a high performance Java engine that can run both on our standard JVM being J Rocket or it can run on the advanced JVM offering around WebLogic real time giving a much more predictability and low latency. But again, I think what’s important about this is that it is a full application stack that allows somebody to be able to build very quickly and event processing environment and has all of the aspects of whether it’s going to be scale and volume complexity or whether it’s going to be a temporal aspect.

So, I’ve got on the slide now a sort of architectural view, I guess, of what we call TED is the time and event driven products. Basically, what you’ve got in the middle as the WebLogic event server and then underneath it, sitting on, really the event server sitting on top of WebLogic real time or J Rocket and the difference between the two is WebLogic real time offers both determinism and low latency and I’ll talk about that more as I go forward and give you a use case on it, but really drawing this to the fact that this is the only Java application server for high performance or high volume real time complex event processing and we have built this on a light weight Java application server so this isn’t J2EE specific that people would normally look at, this is actually Java.

We felt this was a lot more pertinent to this market space being a sort extreme transaction processing end of the business and then we have EPL as the standard language which is kind of like an SQL but it has temporal aspects in it so the things like normal linear events and object based events, it really helps people for that and then you can do things like throat scheduling and things, time critical streaming on that as well, so I think, again, what we’ve done is tried to build a platform to allow somebody to rapidly execute against an event application. And, you can virtually do the development in your favorite tooling language or development language whether it’s Springframe frame or Simpogio and then we use BEA Workshop as the eclipse space tooling for this as well.

And, again, we offer two JVM’s whether it’s J Rocket being high performance or it’s WebLogic real time being extreme performance. So, WebLogic real time is really around low latency and high performance so 10 milliseconds and less than a millisecond as an average response, 100% guarantees and it’s basically instantaneous so it automatically inherits the values that you have on your applications. Moving forward on the event processing, really, you have SOA as a standard way of delivering events into the business but you need more dynamices around it and so what we;ve found that by including things like stateful unpredictability and scalability work you roll in the event server and then you actually have a data stream optimizer that hangs in so, you know, what event driven architecture does link with SOA is almost like SOA next.

And then, finally, I have a couple of quick examples. One is a customer example of the equity trading house which is really around a client that has some specific syntax around stock price movements when the 3% spread exceeds the, you know, “XX” and this is very specific to their market space but really it can only handle 10,000-20,000 concurrent signals and by pushing this onto the event server, they went up to 50,000 so that means that they actually have a better, clearer view of the data coming into their business and they can be a lot more competitive.

And then, finally, an example of a trading house that was using WebLogic real time and prior to that using an arbitrage, they were losing around $200,000 dollars per day on their arbitrage solution. Now, they installed it as standard on top of WebLogic real time and within 3 days they decided that they were going to purchase WebLogic real time so they automatically inherited the values that you’d get form WebLogic real time, it negated any of the competitive disadvantage and they’ve rolled forward. And, again, important to add that these are unique to the business, whether it’s the platform that we have around event server, whether it’s WebLogic real time with low latency and guarantees and automatic inheritance but again, these are phenomenal technologies that we believe is quite and exciting for people to look at moving forward. So, with that said, I’ll hand back to Beth for any questions.

BGB:    Thank you Guy. I’d like to remind everyone you can send me your questions by pressing the “Ask a Question” button and before we go to your questions, we have a question for you. Are you intrigued by or convinced of the value of an application server for event processing? Please take a moment to answer the question while we read over your questions.

Roy, given the definition of complex events would not composite event be a more accurate definition?

RS:    Well, that’s an interesting question. The term complex events did actually become the most common use of the term when you’re talking about an event that signifies a number of other events. Now, composite event was used by the active database community in the 1990s and it had a slightly different meaning. It meant a complex event in which you actually had a copy of the data from some of the member events so because the word composite event had a specific meaning and for this community, if you use that as a general term for complex event, you could end up with some confusion.

So, complex event is the term that is being used although of course, it’s ironic that a complex event is actually simpler to deal with than a simple event simply because in a simple event you might need hundreds of them to convey the same information that you can convey with one complex event. So, complex event is the term, a general term, that is used for an event that summarizes other events.

BGB:    OK, now can you, you mentioned the government, are there other industries seriously planning for EDA now?

RS:    Absolutely. Well, there’s really two answers to that question. If you look at industries that use the term event processing and complex event processing, that’s a fairly small community of developers at the present time. It would include military intelligence, the capital markets and a number of other places. But a lot of companies are doing complex event processing but they’re calling it business activity monitoring or they’re calling it by the name of their application. So, really supply chain applications and many of the other kinds of monitoring applications that exist out there are doing complex event processing whether they call it that or not.

BGB:    OK. Now you gave figures on typical business event volume. These would be correlated into complex events. Do you have similar figures for the number of complex events that a large organization might have defined?

RS:    Yeah, most of that 104 which is 10 thousand or 107, 10 million business events per second, most of those are simple events. Now simple and complex are in the mind of the beholder. It’s kind of relative. Any simple event, you can break down into a yet simpler event so you can always decompose anything into a smaller unit. A simple event is something that hasn’t been decomposed, for which there is no sensible reason why you want to go any more fine grained. That’s the only difference really between a simple event and a complex event. Of the 10,000-10,000,000 business events per second, most of those are simple events and probably it would be an order, maybe two orders of magnitude, maybe 100 to 1 simple events vs. complex events. So you could be looking at, you know, you could be looking at tens of thousands of complex events per second already being recorded and transmitted in a large enterprise today.

BGB:    OK. Now, Guy, there are a number of very specific questions about the product coming in and maybe I’ll have you answer some of these off-line. But, can you please explain the difference between the offering of CEP vendors and BEA’s offering?

GC:    Yeah, I mean, in essence we have the complex event processing inside of the event server. What we felt is the same way that BEA normally approaches the market from the IT development perspective is we wanted to provide a more comprehensive, integrated platform so you have both complex event processing and things like with a bunch of real time integrated into it where you have near real time so we can handle more aspects around scale and quality and complexity and latency so it’s really a time to market, fast development platform that allows people to sort of have more of the code integrated into a single integrated, you know, unit.

BGB:    OK. Now, Roy, a consultant writes in saying “Am I too attached to the request/reply model when I think about how to reconcile event driven operations or is it verification of expected action part of the feedback of the event processing model?”

RS:    Well, in the event processing model, in some situations, you do need feedback, some sort of acknowledgement so that the sender of the event object knows that it has been received. In many cases you don’t. So, in the stock market example, there is not acknowledgement that goes back to the Exchange or to the data feed provider that says, I’ve gotten this stock pick. So, for many types of events, you don’t want acknowledgements. Now, if you’re sending a payment or some sort of transaction and you need the data integrity, you have a choice – you can do it with the request/reply model, which is often done or you can do it with a push model which is not event driven or you can do it with an event driven push model. In that case, you may want to still send back an acknowledgement message and you can do that – an event driven architecture can have a cyclical directive graph so something comes back to the event source and that would still be event driven even though there was an acknowledgement coming back and it wouldn’t be request/reply. It would still not be a method or operation name involved which is one of the key attributes of the event driven architecture.

BGB:    OK. And just to get clarification on terms and technologies, when you use event processing in a straight through process, is that the same as publish and subscribe?

RS:    Well, that's funny, I think a lot of times people are already using publish and subscribe for event driven architecture but you don’t have to. You can do event driven architecture without doing publish and subscribe. You can use any form of one-way messaging to implement event driven architecture. But it’s true that a lot of people that are doing event driven architecture in the past or message driven processing in the past called these pub-sub systems – publish and subscribe systems so there’s a high overlap but you can do publish and subscribe without doing event driven architecture and you can do event driven architecture without doing publish and subscribe. So, an architect really should be careful to make sure that they are thinking of whichever variation is appropriate for that particular use case.

BGB:    OK. And another clarification, presumably by complex you mean lots of inputs very quickly but wouldn’t that be more like event stream processing where you’re filtering, I mean, is there a difference? How do you define that difference between complex event processing?

RS:    Well, event stream processing is, I think, always complex event processing. It’s a type of complex event processing because when an event stream comes in, the stream coming in is probably simple events in most cases, although they don’t have to be and then from those, you’re looking at multiple events to find a pattern of some kind, and you’re aggregating or you’re correlating, you’re filtering and so forth and so what you get out of event stream processing is almost always a complex event. So…

BGB:    OK, but there are some complex events that you have streams coming from different activities that would come in and you sort of have to correlate it against patterns, it’s not just like filtering tens of thousands of events coming in for one type of event like stock price.

RS:    Exactly. Event stream processing has a connotation that what’s coming in may just be events of one type and you’re finding a pattern in those events although any commercial product that does event stream processing is capable of finding relationships across more than one event stream. Still, some of the kinds of complex event processing may be looking at lots, dozens, of different kinds of events and that, in most cases, people wouldn’t use the word event stream processing for that kind of work but there really is no hard and fast definition or white line between event stream processing and other types of complex event processing. So, it’s best really to look at the usage patterns in the particular application so you can understand what’s happening and then not put too much weight on either of those titles.

BGB:    Now, do you see a need for a search for already defined complex events if you, you know, are subscribing to them? How do these systems work?

RS:    Well, I think it all depends on the business situations. All of these things are meant to be tools to help you make faster and better decisions. So, in some cases, to build a business activity monitoring application, you can leverage some existing sources of complex events. In other cases, the only sources of events you have are simple events and then you are going to have to create the rules by which you start with simple events, create complex events then perhaps you want to build a hierarchy. You may then correlate multiple complex events to get yet a higher level of complex event so you may have a hierarchy where you’re cascading events and getting them more refined, summarizing significance on yet higher and higher levels as you go.

BGB:    OK. Now, Guy, does BEA have any thing to manage latency in the real time application?

GC:    Yes, so integrating into WLR2 which is the WebLogic real time we have a latent analysis tool and this enables you to take the pulse spikes and drill into them in production which is again, completely unique to the BEA offerings. So, yes and yes.

BGB:    OK. We are running out of time so if we don’t get to answer your question, we will take them off line but there are a number of questions, Roy, about where we are in the market, why is it a growth sector in Gartner’s view and, finally, is there any specific limitation in the CEP concept perhaps that will eventually be overcome as the market matures?

RS:    Well, we certainly see a lot more activity in this market than existed five years ago. A lot of this technology just was developed in academia in the late 90’s and early 2000’s. The commercial products are generally less than several years old and sometimes within, you know, the last year. And a lot of this comes about because people suddenly are finding that things that they always wanted to do, they always wanted to have, more up to date information about certain things, they’re now finding it practical to do.

So, I think it will take between 10 and 20 years for this to be largely exploited within companies
10/10/2006 CIOaudio - John Halamka, CIO of Harvard Medical School and Care Group Health Care Gold Club Protected
Listen in to CIOaudio, as Elizabeth Book interviews John Halamka, CIO of Harvard Medical School and Care Group Health Care. Find out how John is an "early adopter" of RFID technology!

Explore Our Topics

  • Virtual Conferences
  • Webinars
  • Roundtables

BPM in Action

March 10, 2011

The sixth annual BPM in Action 2011 Virtual Conference will explore cutting-edge market developments in BPM and describe how to leverage them for improved business operation and performance. More

View All Virtual Conferences

Smart Case Management: Why It's So Smart.

Date:Nov 05, 2009
Time:12:00 PM ET- (17:00 GMT)


Date:Oct 29, 2009
Time:15:00 PM ET- (19:00 GMT)

View All Roundtables
  • Research Library
  • Podcasts
  • News

Joe McKendrick: Part II of II: Designing Evolve-ability into SOA and IT Systems

In part two of Joe McKendrick's recent podcast with Miko Matsumura, chief strategist for Software AG, they talk about how SOA and IT systems need to change and grow and adapt with the organization around it.

Listen Now

Phil Wainewright: Helping Brands Engage with Social Media

Phil Wainewright interviews David Vap, VP of products at RightNow Technologies, and finds out how sharing best practices can help businesses understand how best to engage with online communities.

Listen Now

Peter Schooff: Making Every IT Dollar Result in a Desired Business Outcome: Scott Hebner of IBM Rati

Scott Hebner, Vice President of Marketing and Strategy for IBM Rational, discusses a topic on the top of every company's mind today: getting the most from IT investments.

Listen Now

Jessica Ann Mola: Where Will BI Fit In? Lyndsay Wise Explains

In BI, this tough economy and the increasing role of Web 2.0 and MDM are certainly topics on people's minds today. WiseAnalytics' Lyndsay Wise addresses each of them in this informative podcast.

Listen Now

Dennis Byron: Talking with...Deepak Singh of BPM Provider Adeptia

Deepak Singh, President and CTO of Adeptia, joins ebizQ's Dennis Byron in a podcast that gets its hand around the trend of industry-specific BPM.

Listen Now
More Podcasts
  • Most Popular
  • Quick Guide
  • Most Discussed

Quick Guide: What is BPM?

Learn More

Quick Guide: What is Event Processing?

Smart event processing can help your company run smarter and faster. This comprehensive guide helps you research the basics of complex event processing (CEP) and learn how to get started on the right foot with your CEP project using EDA, RFID, SOA, SCADA and other relevant technologies. Learn More

Quick Guide: What is Enterprise 2.0?

A lot of people are talking about Enterprise 2.0 as being the business application of Web 2.0 technology. However, there's still some debate on exactly what this technology entails, how it applies to today's business models, and which components bring true value. Some use the term Enterprise 2.0 exclusively to describe the use of social networking technologies in the enterprise, while others use it to describe a web economy platform, or the technological framework behind such a platform. Still others say that Enterprise 2.0 is all of these things. Learn More