The next few posts explore the following design questions: When should a system (or a person) act? Should it act now, based on available, uncertain data? Or should it wait for more data? Should it act quickly based on what it knows or should it act slower to attempt to acquire more information?
Sense and response (S&R) applications deal with uncertainty. Data from
sensors, news stories, market feeds and other sources may be noisy. Adversaries
and hackers may deliberately feed false data to the enterprise. Algorithms that
estimate probable future outcomes must deal with uncertainty in estimates.
Action plans must, perforce, acknowledge that actions are based on estimates
rather than perfect knowledge. All the steps of sense and response apps deal
The degree of uncertainty varies from app to app, but
designers and users of S&R apps should acknowledge inherent uncertainty in
the app. See Scott Schumaker's post: "Dot Connecting in the New Data World" for a related discussion about uncertainty.
Responsiveness: The business benefit of an S&R app is the efficacy of the responses that the app provides. Data acquisition, data fusion, and action planning are valuable to the extent that they help in executing appropriate actions in a timely manner. When you evaluate the business benefit of an S&R app you determine how much better the enterprise responds with the app than using its current operational methods.
The challenge is to execute actions effectively even in
uncertain situations. Indeed, a critical design problem is to determine whether
to: (a) act based on available uncertain data or (b) wait for more data so as
to reduce uncertainty.
For a more detailed discussion see the book by Roy Schulte of Gartner and me Event Processing: Designing IT Systems for Agile Companies
Relationship to the Enterprise Software Stack: S&R apps are similar to other applications in the enterprise software stack. S&R apps differ in the greater emphasis they place on efficacious, timely response in the face of uncertainty.
BI: Business intelligence (BI) applications also deal with uncertainty. S&R apps combine real-time business intelligence with timely response; they are integrated apps that deal with sensing, analyzing, planning and responding whereas BI focuses primarily on analysis.
BPM apps sense the progress of business processes and respond when apps don't progress as expected; in this sense, BPM apps are S&R apps. Sense and response systems monitor the total enterprise and its extended environment - including competitors, adversaries, governments and marketplaces - to determine threats and opportunities. By contrast, BPM focuses on business processes within the enterprise. More business people are aware of BPM than they are of S&R or event processing, and BPM is a more mature business technology. See, for example, Sott Cleveland's post: "BPM from a Business Point of View." The relationship between BPM and S&R is explored in Joe McKendrick's "BPM in Action"
The design question of "act now or wait for more data?" is central to S&R apps but is less critical for BI and BPM apps. Next, let's look at an example that highlight this question, and then we'll analyze two more examples.
An Example App --- Protecting the nation
from adversaries: Applications that sense
and respond to nuclear radiation, chemical, and biological threats are S&R
apps. For the purposes of this note let's focus attention on a single app:
prevent dangerous material from entering the country through its ports.See Container Security Initiative .
The cost of a false negative - for example, not detecting nuclear material in a container entering the country- is horrendous. By contrast, the cost of a single false positive - stopping the transportation of a container so that it can be inspected thoroughly - is much smaller. However, timeliness is critical because large ports handle millions of containers each year, and slowing the progress of containers has important economic consequences. The app must deal explicitly with uncertainty because data from sensors, intelligence agencies, and others sources are inherently noisy.
This app which deals with homeland security may appear to be totally different from the apps seen in your enterprise; however, I'll show the relationship between this app and your business apps in following posts. And homeland security apps highlight the fundamental characteristics of S&R.
Summary: S&R apps emphasize the total end-to-end responsiveness of an enterprise, from monitoring and acquiring data, to integrating the data to detect significant changes, planning to respond to the changes, executing the responses, and monitoring the responses to close the loop. S&R apps deal with uncertainty and timeliness, and app designers often have to trade off greater certainty against faster response. The next posts will explore these issues in the context of applications in different domains.