When it comes to data, the days of equal rights are over.
The traditional thinking around business intelligence and analytics is that every piece of data must undergo thorough transformation and cleansing before entering the data warehouse. Today, this line of thinking has changed.
As BI tools have become increasingly available to all employees for many purposes, the rigid data transformation processes of the past have become overkill for some analyses. The concept of "just-in-time" data allows for the use of native data in a growing number of circumstances. Adopting this more flexible approach to BI can help organizations achieve broader adoption and faster results from their systems.
Let's start by taking a closer look at how companies have approached BI in the past.
The notion of ETL
Extract, transform and load (ETL) and data-quality processes have been used for years to bring disparate data together into a consistent format that supports end-user dashboards and analysis. The transformation aspect of ETL involves reformatting of the data, cleansing to remove duplicate copies and inconsistencies and integration on a common platform. This requires a significant amount of time from skilled developers and establishment of daily processing requirements so that reporting systems are up-to-date the next day. ETL is required for many types of analyses, but it's expensive and creates delays for all users, perhaps unnecessarily.
Some limitations of ETL include:
- BI for the few: To support these additional processes, BI has traditionally been owned and controlled by IT workers who oversaw the sourcing of data and creation of reports, and/or by a handful of analysts whose full-time jobs were to prepare reports for management. This means that most employees don't benefit from better decision-making because they can't easily access data or create their own reports. Companies then wonder why they don't have a stronger ROI from their BI systems.
- Volumes of data, slow analysis: Many organizations possess a total data volume in the hundreds of terabytes, and even approaching the petabyte mark. Yet it can take analysts hours, or even days depending on the backlog of requests, to deliver useful reports. These days, that's too slow for many competitive industries.
- As Chuck Schweiger, business analyst with retailer Timbuk2, recently remarked in a case study: "Our [enterprise resource planning] system has a wealth of data, but there really was no easy way to get that data out and interact with it. Moreover, our prior reporting and analytics process was so full of handoffs that it was not a workable solution for today's on-demand, flexible, global business environment." Timbuk2 took a Software-as-a-Service BI approach to overcome this barrier.
- Lost data, lost opportunities: Time and budget may not be available to transform all data, and ETL tools have a tendency to transform data beyond analytic needs. In other words, the actionable qualities observed in the data's original state may be lost as it becomes sanitized and rationalized for a higher-level database.
-1-