Closing the decision loop

My pal Madan asked an interesting question over on the BI in Action blog. He was discussing the problems of "predictable" marketing and asked how to:

"manage post-analytic decision maintenance for smarter decision making. Or in other words how predictive analytic systems can be more self-learning based on their success or failure, and adjust decision strategies accordingly. Without that critical feedback loop it's all really just predicable analytic marketing"

The solution to the problem involves two things - more models and something called "adaptive control". More models first. As companies become more sophisticated users of analytics they will find they can built many models for a prospect or customer. The likelihood that this offer will attract them, the likelihood that they will apply for and activate the offer, the likelihood that they will not pay their bill on time and so on. These models must be traded off against one another - I might mail someone who is low risk even if they are unlikely to accept and not mail someone who is high risk even though they would accept in a heartbeat. Effective use of information collected about how customers and prospects respond to offers can be used to build a wide variety of models. In Madan's case it is not enough that he has a high likelihood to need moving services (one model), he must also have a reasonable likelihood of actually buying such services when offered (another).

Adaptive control is actually the more important concept here. The challenge with decisions is that customers (and others) respond to the decisions you take outside your system. You may or may not get a response that tells you what they are thinking. The problem is that if you make the same decision every time (always sending more mail, say) then you only learn about their reactions to that particular decision. Adaptive control involves setting up your systems to be able to make one of several decisions. One of these is typically the "Champion" or the one you think it most likely to work and the others are "Challengers". A small percentage of your transactions are then routed through the challengers and results tracked. The idea is to learn about reactions to other possible decisions (stopping mailing after the first is ignored or stopping mailing after, say, 5 have been ignored). If one of the challengers does better than the champion then it becomes the new champion and you start testing again with new challengers. You might also find that a subset of those being tested give better results for a given challenger and this would allow you to design a new segmentation approach to make sure the people who fell in that subset get treated that way. You test, you learn, you refine.

So why don't companies do adaptive control more often? Well sometimes it is a question of not being willing to test any customers with what might turn out to be a worse approach (risk avoidance), sometimes it is arrogance (this is the best approach, why would we try something else) and sometimes it is a problem with IT people not realizing that decisions are different. No amount of design, requirements management or investment of any kind will ensure that a decision is optimal forever. Decisions must be made to change easily and often. In this they are unlike most other kinds of "code". Without a mentality that decisions are different you cannot build an adaptive control mindset and without one you will not justify the necessary software, process and data investments. This is why we talk about decision management not just decision automation.

I blogged about adaptive control over on my other blog if you are interested.

How'd I do Madan?

P.S. there are some odd rules in the credit world. Essentially, and don't quote me on this, if you pull someone's credit information as part of a marketing campaign you must offer them credit. Companies try and define a good target segment and then only pull the records for that segment but, having pulled it, my understanding is that you have to get some kind of offer. In other words they may know you are not going to accept but, because you fell in the category they pulled, you have to get one. Or something. That said, Madan's problem is clearly a real one as it occurs even where no such legal issues exist.

Technorati Tags: adaptive control, analytic application, automated decision making, BI, business rules, decision, decision automation, predictive analytics