We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

New Frontiers in Business Intelligence

Nari Kannan

Five Root Causes of Bad Data Quality in Business Intelligence

Vote 0 Votes

In the age of self-service Business Intelligence nothing could put off users than realizing that 15 to 45% of operating expenses of organizations are spent on cleaning up Bad Data!

Here's an article about that topic - The Data Quality Wager.

In my mind, root causes for Bad Data Quality can be many, but here are five that jump out as fairly obvious ones:

a. Bad Software Design and Implementation:  Data capture is rarely implemented in many transactional systems with well-thought out validation. At the field type level where a numeric field should not be accepting other characters, or at the next level where the semantics of the field dictate whether the data is good or not. Telephone number entry, for example. Then at the form level, where validation may need to be made at the backend with another table. If I enter a customer name, can I find and have the person validate whether that name and address and may be a phone number match, before typing the rest of the stuff in?

b. Mistakes in Extract-Transform-and Load Programs:  The data may be good in the transactional system but transformational rules applied in the ETL layer may be changing and creating bad data.

c. Bad Business Processes -  If I had a penny for everytime someone in a store or a business had to call for help and they showed the person entering the data how to override some check and essentially find a shortcut to force the system to accept the data, I would be rich. There may be legitimate cases where these overrides may be needed but only some kind of ongoing audit can fix some problem like this. Bad data caused by bad business processes. Happens all the time!

d. Inadequate Training on the Software Systems - It is a reality in many companies that the software implementation drags on and is late and they cut short the time, attention and effort needed on the training on its use. Then complaining about bad data is moot!

e. Errors in the BI Reporting Logic - The data warehouse may be correct in its data but the reporting tools may have bad logic and hence the user sees bad data as a result! Many companies do a systematic verification of all the steps from the origin of the data to all its transformations but many do not.

Bad data causes many heartaches but unless they see this as a systematic end-to-end process of creation and consumption of data, root causes may never be fixed, unfortunately!

Thou preparedst room before it. and didst cause it to take deep root. and it filled the land. The Bible


Nari Kannan's blog explores how new approaches to business intelligence can help organizations improve the performance of business processes--whether these processes are creative or operational, internally-focused or customer-facing, intra-departmental or across functions.

Nari Kannan

Nari Kannan started and serves as the CEO of appsparq, a Mobile Applications development company based in Louisville, KY with offices in Singapore and India. Nari has over two decades of experience in computer systems development, translating product and service strategy into meaningful technology solutions, and both people and product development. Prior to this, he has served as both Chief Technology Officer and Vice President- Engineering in six successful startups, two of which he co-founded. He has proven experience in building companies, engineering teams, and software solutions from scratch in the United States and India. Prior to this, Nari started Ajira Technologies, Inc., in Pleasanton, CA, where he served as Chief Executive Officer for more than six years. While at Ajira, Nari was instrumental in developing service process management solutions that modeled, monitored, and analyzed business processes, initially targeting the Business Process Outsourcing (BPO), Telecom, and Banking verticals in India, and Finance, Insurance, and Healthcare verticals in the United States. Prior to this, he served as VP-Engineering at Ensenda, an ASP for local delivery services. He also served variously as Chief Technology Officer or VP-Engineering at other Bay-Area venture funded startups such as Kadiri and Ensera. He began his career at Digital Equipment Corporation as a Senior Software Engineer. Nari has a long involvement with Customer Support and other customer facing processes. At Digital Equipment Corporation he was involved with their 1800 person customer support center in Colorado Springs, Colorado. He was tasked with coming up with innovative tools to help customer support people do their jobs better. He holds a U.S patent for a software invention that automatically redirected email requests for customer support to the right group by digesting the contents of the request and guessing at which software or hardware support group is best equipped to handle it. At Ensera, he led a 45 person team in developing an internet based ASP service for handling auto insurance claims, coordinating information flow between end-customers, Insurance companies, Repair shops and Parts suppliers. Ensera was acquired by Mitchell Corporation in San Diego. Nari holds a B.S. degree in Physics from Loyola College, and an M.B.A degree from the University of Madras in Madras, India. He graduated with a M.S. degree in Computer Science from the University of Massachusetts at Amherst in 1985. Contact Information: Nari Kannan. Email: nari@appsparq.com Mobile: 925 353 0197. Website: www.appsparq.com View more .


News Feed

Recently Commented On

Monthly Archives