Data at Your Service

In last week's blog, we talked about the value of Master Data Management (MDM) and how MDM can provide a consolidated view of key information inside the company such as the customers, products, or suppliers. This week, we will look at another alternative approach to providing a "single view" of important business information, which is one of the most common requests received in an IT department. Typically, the marketing or sales organization will start an initiative to cross-sell or up-sell other products or services into the existing customer base. For example, your mobile phone provider might want to tempt you into upgrading your plan, buying an additional phone, or signing you up to a multi-year commitment. If you're the mobile phone company, in order to carry out this initiative, you would obviously need to have a view of your current customers. Easy - right? Typically, small pieces of information about customers are littered throughout the data center. For example, fragments of customer data might be spread over the billing system ("the" accurate address), the CRM system ("the" accurate contact number), the marketing automation system ("the" accurate email address), the customer support system, the credit system, the loyalty program system, and other more obscure places. And, further consider the complications that arise when companies merge, when orders are placed from the same company but from different offices, when customers move/get married/go chapter 11, etc.

At a high level, the IT challenges are as follows: fragments of the customer's data might be in different locations, customer information is likely to be duplicated and different, and bringing the information together into one actionable piece of information is just ridiculously difficult. Of course, most any IT problem can be solved by a simple matter of typing (read: writing tons of code) and that's what IT organizations many times do. Trying to write a ton of code to solve the integration needs of one business problem seems fairly straightforward, until you see that you've done this for project after project for the last several years, and now your IT infrastructure is so brittle that changing anything almost certainly breaks something.

MDM provides one alternative to bringing together this diverse information into a one-stop shop for your customer data for real-time applications- although MDM is typically very expensive to implement (but could be well worth it). The relatively new market of data services provides a few other ways to solve the single view problem. Data virtualization (or data federation) technologies approach the problem in two steps. The first step is to identify what attributes of a customer's data are required (and available). For example, company name, billing address, shipping address, primary contact name, phone number, email address, purchase history, lifetime value, date of last customer support call, etc. Typically, a data architect will go through the process of identifying what systems supply the most trusted pieces of this information. The data architect will then create a data model within the data virtualization software to define these key entities and create a physical mapping of where the information exists (or how it can ultimately be computed). For example, the customer account number comes from the SAP system, the customer contact is in Salesforce.com, the last support call can be obtained from Peoplesoft, and the lifetime value can be found by summing order totals in the homegrown Oracle billing system. In summary, the first step is to define where the data virtualization software can find the information when requested to do so (this is sometimes called creating the data model, the common model, or the business object model).

The second step is to retrieve the information from these disparate systems real time when requested - that is, to virtualize the data when requested to do so. For example, a customer service rep will want to receive the customer's contact information, what products/services they own, a flag to indicate if they've paid their bill, and a brief history of their prior call history in the last 12 months presented on one screen immediately when he or she receives a customer call. The data virtualization server will process a customer query in real time to aggregate and filter the results from a diverse set of IT systems, and then provide the results. The query may come in the form of SQL or some other industry standard query language depending on the specific implementation of the data virtualization product.

With data virtualization products, the IT staff becomes more agile because new business-line projects don't involve writing a ton of code to glue systems together in a point-to-point manner. Instead, new projects are quickly implemented because the IT staff is focused only on the business logic required and not the integration work that needs to be done to bring together the right view of the information. The job of data integration is done by the data virtualization product, and the mapping leveraged by this type of product is reusable from project to project. Additionally, using data virtualization products reduces the complexity of changing the IT environment. For example, if we want to migrate from Siebel to SAP, then the existing applications don't need to change. Instead, we simply change the mapping inside the data virtualization server of the customer name, (for example, from Siebel to SAP) and our existing applications will run unaltered.

We've now discussed two types of data services that provide a "single view" of our most important business information. MDM systems are well suited for complex environments where information is not trusted (i.e. duplicate entries may exist for IBM and I.B.M.). Data virtualization systems are well suited for integration across systems where there is a trusted information source, and where data is needed real time, but the exact type of data is not known when the application is designed (i.e. information will be requested and updated via queries). Next week, we'll look at another type of data services that solve the single view problem, called semantic integration.