We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

ebizQ's Business Agility Watch


The Coming Revolution in Predictive Analytics: Podcast With Norman Nie

Vote 0 Votes

Listen to my podcast with Norman Nie, the CEO of Revolution Analytics. Norman is the co-inventor of SPSS, the predictive analytics company that was recently acquired by IBM. In this podcast we discuss predictive analytics and how open source R statistical language just might revolutionize it.

Listen to or download the 10:51 minute podcast below:

Download file


PS: Now, can you provide me with a quick overview of predictive analytics and what your company is doing in this space?

NN: Yes, predictive analytics is about using statistical models to understand where your customers are, what you're employees are doing, how well you're performing and quality control, virtually all aspects of enterprise activity, business activity, and business decision making. There's come to be an intellectual consensus that data drives out armchair thinking and that we now are in a period where people have the training, the background where they can use predictive analytics to run their businesses more efficiently and more competitively.

There seems to be a great storm. "R's" role in that storm has been enormous in that it is an open source invention of several biostatisticians and it literally has exploded itself across all of the disciplines of the academy and makes it where statistics are used which is now basically everywhere except for literature and the arts. So there is a wave of growth and explosion of issue of this that it began slowly 40 years ago when I was involved in the invention of SPSS and now has culminated in a whole new wave and "R" seems to be running the leading edge of that wave by having two million users that are spontaneously emerged particularly from teaching, and research in the academy, and now spreading into business to fulfill this new consensus.

Now, what new industries do you think can stand to actually benefit from predictive analytics?

I believe that you're going to find that virtually every industry there's enormous horizontal compatibility between statistical methods across areas of business which is one of the reasons why you look at the existing legacy products out there like SAS and SPSS IBM and they are involved in businesses from everywhere from keeping track of time-to-part failure in the NASA Shuttle, real-time prediction of stock transactions. So there's a ubiquitous (indiscernible) involved in methodology and a transfer is a specific algorithm's one field to another.

Take for example the fact that we once had a set of things called survival analysis they were for cancer patients. They're now being used to do mean time to failure in manufacturing parts so these things grow across fields and in multiple applications. Leading edge is (indiscernible) they're finding in pharmaceutical, in life sciences, and in high finance. It's sweeping across the entire sphere of businesses.

Can you give me some information on open source "R" statistical language and what is exactly so revolutionary about it?

Well, I think it's a fascinating story and a great, great tribute to the open source endeavor. As I said in the early 1990s, several biostatisticians including Robert Gentleman who serves on our board put together this language. It is the most complete and versatile, and flexible statistical language in the world. There is no statistical expression, which cannot be accurately and correctly rendered in the "R" Language. It has no built-in pre-packaging of procedures as the legacy products do but packages get written by individual users and then shared. So it has the largest statistical programming staff of any organization in the world. There are 2,500 "R" packages out there and two to three million users out there regularly using it. It has also taken over teaching the function at major universities.

As we've built over the last 40 years, this understanding of the use of statistics in a wide variety of fields. Now in the generation, faculty and lead graduate students are going to push the boundaries of predictive analytics and therefore switched the "R" so that they themselves can control rather than be controlled by preexisting procedures.

How are you enhancing "R" to be the foundation for a new predictive analytics?

The role of Revolutionary "R" is to provide commercialization or enterprise level support for the open source product by means of dynamic loading and plugs into sockets that make the product enterprise ready. We are, for example, providing high-speed big data facilities, which we'll release later this year. They will make "R" several orders of magnitude faster and be able to handle terabytes if not hetabytes of data by putting a new file structure on top of "R" a high speed multi-processing chunking of data across facilities, 14 new statistical procedures which are especially geared for high volume work in preparation for this deluge of data which is coming at us in the form of terabytes and hetabytes and which the legacy software will not be able to do.

So I think we're providing rapid user interface. A comprehensive one, an extensible one that will enable beginners to more easily learn "R", middle range people more easy to use "R" and even the most experienced programmers of "R" to move back and forth between the language and the user interface, increasing productivity to the top end, and accessibility at the bottom end which is absolutely necessary to be ubiquitous in operation. Things we are doing here are things that really cannot be done with the existing legacy products, which after all core engines, were written 40 to 50 years ago.

Can you give a real world example of an actual customer using "R-based" predictive analytics?

Sure. Alliance Bernstein which is a finance organization doing a very, very demanding analysis of large numbers of transactions. We were asked to come in because they were having problems. They were running some programs of legacy vendors, one of the major actors in the field that were taking nine hours. And we brought them in, in our Alpha release version of our high-speed and big data and those are now running in 15 minutes. And as we complete the rest of that high-speed, huge data model, they'll be running this problem in matter of under a minute. And we're now in Alpha testing stages with a large number of companies who are requesting similar kinds of things.

There is, of course, some skepticism with predictive analytics insofar that it's so difficult to actually predict things. Now, how do you answer that and what actually do you see for the future of predictive analytics?

There's a fundamental notion in statistics; it's the essence of statistics. It's the proportional production of an error. And yes, we understand particularly when dealing with human beings as units of analysis and social systems, and economic systems and biological systems that there's an enormous amount of unexplainable phenomenon at the current level of understanding.

The job of predictive analytics is to proportionally reduce that error, an error that is at its highest with armchair theorizing unverified by facts. And when we continue to build those real-time models, we're constantly changing and monitoring the way what our customer relationship management or quality control we keep building models that further and further reduce the error. And therefore, nothing is perfectly predictable in these open statistical systems but every time you reduce a few percentages of errors you become more efficient, more competitive, a high quality producer and distributor of goods and services.

What do you see for the future of predictive analytics?

Gardner and others see predictive analytics currently as a $7 to $10 billion business and that predictive analytics and its associated business intelligence organizations practices are growing at 25% a year. This is a wave of the future. This is the way enterprises, government agencies, corporations, research projects, product development - this is the way in which it's going to be understood and the way in which you make it incrementally more efficient.

ebizQ’s expert blog team covers a broad range of BPM, business integration, business analytics/monitoring, collaboration, content and related issues.

Peter Schooff

Peter Schooff is Contributing Editor at ebizQ, and manager of the ebizQ Forum. Contact him at pschooff@techtarget.com

Recently Commented On

Monthly Archives