Instantaneous reaction to real-world event data can make a significant difference in a variety of applications, including network intrusion detection and prevention, airplane flight control, identity theft prevention, credit card fraud identification, eBusiness transactions, and stock market trading.

For instance, in financial services more data is being generated faster than ever before in stock exchanges, in part because of global market and regulatory changes. Traders who can run quantitative analytics and automatically trade from high-volume, high-speed data feeds gain critical competitive advantage. In telecommunications and networking, as the number of customers served and their usage of system resources explodes, maintaining system uptime through split-second response and remediation to network intrusion or breakdown becomes a critical necessity. And, in government and military applications, access to critical complex analytics in real-time can make the difference between survival or defeat on the battlefield, or can help avert domestic or international terrorist attacks.

The need to react and respond instantly to high-volume, high-speed data for competitive advantage, to minimize risks, and avert disasters has never been greater. Thus, a revolutionary new technology is needed to perform complex analytics on high-speed, high-volume data to enable instantaneous response. That new technology is high-performance Complex Event Processing (CEP).

High-performance Complex Event Processing

High-performance Complex Event Processing is the continuous processing, analysis, and response to high-volume, high-speed data streams in real-time. It can be used to identify and analyze cause-and-effect relationships among events, and because it does all of this in real-time, it enables instant and informed responses to critical intelligence in real-time.

Four different approaches are typically considered for high-performance Complex Event Processing:

• Custom-Coded Applications are typically written in Java, C++ or other similar programming languages. Until the recent advent of off-the-shelf CEP, this had been the leading solution for many organizations. However, it is limited by long development cycles (e.g. 6-9 months or more), high cost of maintenance, and inflexibility.
• Rules Engines watch input data streams for any conditions of interest that match its rules - typically condition/action pairs, usually expressed using "if-then" notation. The rules are typically written in a proprietary language, and when a condition of a rule is matched, the rule is said to "fire" a corresponding action, alert or output to external applications.
• In-Memory Databases reduce the latency delays of traditional database management systems by eliminating disk I/O, caching, and transaction logs, but they still incur latency penalties and require custom-coding to build CEP applications.
• Stream Processing Software is specifically designed to handle streaming data, performing queries and computations on incoming messages as they fly by, without storing them. Stream processing uses specialized primitives and constructs, such as time-windows, to express stream-oriented processing logic.