The latest issue of Teradata Magazine came to me today. This had a special section on Active Enterprise Intelligence and had three interesting articles - The active advantage, The moment of truth and Active data warehousing: from nice to necessary.
The gist of these articles is that "active data warehousing brings sophisticated insights to day-to-day business decisions". I agree but I think there is a need to add a decisioning infrastructure, especially one based on business rules integrated with predictive analytics, to turn a typical enterprise data warehouse into an active one. Most data warehouse / BI projects are focused on delivering insight to people, typically off-line (albeit using increasingly up to the minute data). To bring the kind of insight people get from these projects to operational systems you need a couple of things:
- For sure you need a data management/data warehouse infrastructure that can support the kinds of users, mixed workload and volumes/response times involved. After all you are going to be driving a much higher volume of decisions once you apply this to operational decisions.
- You need a rules management infrastructure so that you can define what actions to take in response to events or requests and so that the folks who understand the business can be part of defining how to respond
- You need to be able to bring predictive or executable analytics to bear in these rules and processes so that the probable future predicted by the data you have to date can be allowed for
- You need to get the time from new data to new decision to be the smallest possible by carefully thinking about where and when to execute models, rules, data analysis etc.
On this last point I have long been a fan of Richard Hackathorn's view of latency. He uses the concept of the value-time curve. This curve is designed to show that the longer you take to respond to new data, the less value there is in your response (assuming you make an equally good decision).
Dr Hackathorn describes response latency as consisting of the sum of "capture latency, analysis latency and decision latency". As he says "As quicker actions are taken, we move up the value-time curve, increasing the value gained". While he believes that it is hard to reduce decision latency due to cultural issues, I have seen many companies use business rules approaches to automate decisions, at least in a high percentage of transactions, and so reduce their decision latency close to zero. This is why I think a decision/rules management infrastructure is so important when trying to deliver active data warehousing.
Lastly, the comment was made in one of the articles that "an EDW supporting active data warehousing serves as the brain of the enterprise". I would say more that an EDW serves as the memory of an enterprise and that the associated decisioning infrastructure serves as both brain (deciding what to do) and as part of the nervous system (making it happen).