We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

New Frontiers in Business Intelligence

Nari Kannan

Fixing the First Root Cause of Bad Data Quality - Bad Software Implementations

Vote 0 Votes

My last blog entry Five Root Causes of Bad Data Quality seems to have struck a nerve with a lot of readers in cyberspace. I got a lot of re-tweets.

I thought I could expand upon each root cause and some good ways to fix them. Here's the first one: bad software implementations!

Everybody has their own story about software implementations lurching or stagnating, then going forward at incredible speeds when deadlines are near, with bad software quality ensuing. An implementation can pass all the quality assurance tests, but bad data quality is not routinely among those tests. 

The primary contributor to bad data quality is not just field level validations, form level validations and semantic validations.

Just to recap, field level validations need to be designed into the form itself to check, for instance, whether you are accepting alpha inputs into a numeric one. Form level validations do things such as checking the city against the zip code or filling up required fields. Semantic validations might check whether a customer number that has been entered is actually available; the information being entered is checked against a backend database and then accepted.

But bad data quality could also arise due to not designing all the possible use cases into your software. When you enumerate your users and use cases, have you allowed for generic exceptions? Are the generic exceptions used only rarely? If generic exceptions are used 50% of the time, you can see where the bad data is creeping in. Your shortcuts for exceptions are  being overused and spoiling the data you have. Exceptions need to be just that--exceptions.

Another root cause of bad data quality is the ensuring of single identities within your system. Do you have two or more accounts for the same customer? Happens all the time, right? Especially if they two accounts are in two different divisions of the company and no one knows that they represent the same person.

Master data management (MDM) techniques and software can be implemented on top of existing systems to address some of these issues.

Meanwhile, the problem can be particularly acute if your company has grown by acquisitions. You may be using silos of data that have no connection with each other. Even if they are merged, software development is so expensive that addressing bad data quality and making sure that the data is merged correctly falls to the bottom of the priorities list. 

Bad software implementations also can arise out of not checking out software packages thoroughly for root causes. You may be buying off-the-shelf packages without kicking the tires for bad data quality possibilities. Have the screens and forms been designed to keep these problems out? Have you checked this before you bought the package?

Looking at root causes always helps in fixing problems thoroughly. Spending enough time and energy with evaluating software implementations with respect to their propensity to create bad data always pays off.

It also always pays to look at bad data quality as a result of faulty steps in a faulty process. Avoiding bad software implementations is the first step in fixing this. Fixing other root causes down the pipeline are useless if this first root cause in the pipeline is not fixed.

My software never has bugs. It just develops random features. - Anonymous

Nari Kannan's blog explores how new approaches to business intelligence can help organizations improve the performance of business processes--whether these processes are creative or operational, internally-focused or customer-facing, intra-departmental or across functions.

Nari Kannan

Nari Kannan started and serves as the CEO of appsparq, a Mobile Applications development company based in Louisville, KY with offices in Singapore and India. Nari has over two decades of experience in computer systems development, translating product and service strategy into meaningful technology solutions, and both people and product development. Prior to this, he has served as both Chief Technology Officer and Vice President- Engineering in six successful startups, two of which he co-founded. He has proven experience in building companies, engineering teams, and software solutions from scratch in the United States and India. Prior to this, Nari started Ajira Technologies, Inc., in Pleasanton, CA, where he served as Chief Executive Officer for more than six years. While at Ajira, Nari was instrumental in developing service process management solutions that modeled, monitored, and analyzed business processes, initially targeting the Business Process Outsourcing (BPO), Telecom, and Banking verticals in India, and Finance, Insurance, and Healthcare verticals in the United States. Prior to this, he served as VP-Engineering at Ensenda, an ASP for local delivery services. He also served variously as Chief Technology Officer or VP-Engineering at other Bay-Area venture funded startups such as Kadiri and Ensera. He began his career at Digital Equipment Corporation as a Senior Software Engineer. Nari has a long involvement with Customer Support and other customer facing processes. At Digital Equipment Corporation he was involved with their 1800 person customer support center in Colorado Springs, Colorado. He was tasked with coming up with innovative tools to help customer support people do their jobs better. He holds a U.S patent for a software invention that automatically redirected email requests for customer support to the right group by digesting the contents of the request and guessing at which software or hardware support group is best equipped to handle it. At Ensera, he led a 45 person team in developing an internet based ASP service for handling auto insurance claims, coordinating information flow between end-customers, Insurance companies, Repair shops and Parts suppliers. Ensera was acquired by Mitchell Corporation in San Diego. Nari holds a B.S. degree in Physics from Loyola College, and an M.B.A degree from the University of Madras in Madras, India. He graduated with a M.S. degree in Computer Science from the University of Massachusetts at Amherst in 1985. Contact Information: Nari Kannan. Email: nari@appsparq.com Mobile: 925 353 0197. Website: www.appsparq.com View more .


News Feed

Recently Commented On

Monthly Archives