New Frontiers in Business Intelligence

So far, Business Intelligence has been mired in questions of data collection, data cleansing and just storing the d**g data somewhere, anywhere.

Meanwhile, ask any serious user of Business Intelligence, 9 times out of 10 they may come back with an answer "Yes. The Data is all There, but......".

We have been so preoccupied with the collection and storage of data that the real purpose of all that data has been getting the short end of the stick, all through! The data collection and integrity problems are serious enough to have warranted all that attention, though. Let's not begrudge them that.

Business Intelligence is all about consolidating, aggregating data, and making it provide insights in many different ways. Insights are gleaned from all this data need to be immediate, useful, and easily available to the end users, not IT gurus that know all the magical incantations to tease reports out from all of this.

Previously I have written about how Data Visualization can help a lot with this.

However, Visualization is not all that matters, all by itself. First problem a lot of Business Intelligence efforts must address is that when you are analyzing the data, are the relevant data all in there for you to make use of it.

Funny thing is that when the Data Warehouses were being designed a couple of years back, the company was in a different time and place and since then things have been changing constantly. The question now is has the data you thought you needed at that time is still valid? There is every possibility that you overlooked some things and did not capture some of the data in the detail you needed was being captured and stored.

Simple example - Let's say you want to improve the "Churn" (turnover of customers) factor in a Telecom company. But when you designed the data warehouse you captured only whether a customer was a Residential or Business Customer. Now you wanted to see what's happening with High Value Business customers alone and not waste your cycles on customers that had less value for you.

These kinds of Analysis related issues may come up only at the time of analysis. Now can you go back and finetune your data collection efforts easily so that you can get the precise kinds of analyses that you need?

Life is that way! Sometimes iteration is the only way you will be finding out what is needed at any time, especially in the future.

There might be other newer kinds of analyses, not just visualizations that may be needed, on the fly!

Let's hope the next decade focuses a lot on the Analysis side of things and produce much needed innovations in that area, especially going back and flexibly changing things all the way back to data collection so that you get the kinds of analyses you need, easily, quickly.

Many of the great achievements of the world were accomplished by tired and discouraged men who kept on working. - Anonymous