That's where complex event processing comes in. Complex event processing software helps organizations deal with processing huge volumes of transaction and real-time data-not so much from the transaction perspective, but from the analysis and understanding perspective.

Unlike traditional computer "programs" or applications that operate in a linear fashion, complex event processing systems are designed to manage event-driven information systems-systems where events (such as a financial trade, a banking transaction or perhaps even a retail purchase) trigger specific processing events or tasks. A key goal of a complex event processing system is to identify meaningful events from a large stream of on-going events. Complex event processing systems attempt to discover complex relationships and complex events by analyzing multiple, smaller events.

Complex event processing (or CEP) has historically been associated with banking, financial and e-trading systems, where it's used to process, analyze and react to real-time data. For example, many investor websites need to provide tools for analyzing and responding to hundreds or even thousands of real-time market feeds. A key characteristic of CEP solutions is their ability to query, process and analyze data at fantastic rates-such as 25,000 messages per second, while updating hundreds of thousands of client portfolios at the same time.

But there are also applications for CEP in many other industries and areas. Consider a retail store that wants to focus on its customers and identify ones that have been especially loyal. One way to recognize customer loyalty is to have them enrolled in a customer loyalty program. But what about the customers that aren't enrolled-the ones that might be making repeated purchases at the store even through they're not part of any formal program? Shouldn't there be a way to recognize them and perhaps offer them an extra discount on a current purchase, a coupon or invitation to the customer loyalty program? It's a good idea, but the problem is that it may take a little bit of work to identify such customers.

Let's say the goal is to identify shoppers who have shopped at a store more than four times within a specific time period. In addition, the goal isn't simply to identify them, but to identify exactly what type of customer they've been and perhaps make them offers that are consistent with their purchase history or areas of interest.