Harness big data's power to dramatically improve BPM

The benefits of a solid “big data” strategy are well-known. Solutions such as Hadoop and in-memory data grids offer ways to tackle the volume, velocity and variety of big data for a more agile business. But fewer people are aware that the right big data approach can directly improve business processes. In fact, experts say that process improvement efforts are a prime driver of big data technology adoption today.

"The demand for big data is in process," says Jim Sinur, research vice president at Stamford, Conn.-based Gartner Inc. That's because big data is a vast source of events, triggers, goals and context that can be used to tailor processes to the user experience.

Harnessing the power of big data for better process design and management may sound complex -- and it is. Fortunately, there are best practices for doing the job right. Industry viewers advise a strong master data management (MDM) strategy; a focus on customer experience; and a project management approach to using big data for BPM.

Not surprisingly, an important best practice for any big data initiative is getting a handle on the data itself. The sheer amount of data in an organization can make it difficult to determine which data is actionable--and which is just in the way.

"Sometimes you can be overwhelmed with information and not be able to make a decision," says Clay Richardson, senior analyst at Cambridge, Mass.-based Forrester Research Inc. "So a big part of the opportunity around big data is getting just enough information to make a decision."

When it comes to using big data for processes, the key is to focus on the customer, Richardson adds. By paying attention to the data needed at each point in a customer's journey, businesses can determine what data is critical for specific tasks and transactions. Many companies with successful big data-BPM initiatives use customer journey maps to accomplish that goal. Such maps define all of the places where customers interact with a business, aligning key systems with key data to support optimal process outcomes.

"Put yourself in the shoes of the person getting the task done," Richardson advises. "By taking that mindset, you look at the real interaction points that matter, and how you can use data to actually drive better [customer] engagement."

Oil, gas, utilities and healthcare companies have the best track record for combining big data with BPM because of their strong customer focus, Richardson says. Large numbers of customers, massive volumes of data and reams of historical information make them prime candidates for the approach.

"These industries are trying to use the large amount of data coming in to drive better decisions, and connect that data back to process," Richardson says.

But none of that is possible without a strong MDM strategy, he continues. That means ensuring that the information gleaned from big data sources is clean data: accurate, relevant and complete.

"You want to make sure you have a good master data management-BPM connection," he explains. "That's a basic best practice that we see in companies bringing [BPM and big data] together successfully."

Big data projects require vastly different skills than BPM efforts, which can pose a challenge to business process professionals. For example, knowledge of data cleansing, data analysis and data management are crucial to big data initiatives.

"Process professionals are used to looking at processes and making decisions based on narrow ranges of data," Sinur says. "When you start getting into knowledge work and social network analysis, it takes a different skill set."

To tackle big data like an expert, process professionals should avoid biting off more than they can chew, recommends Jim Harder, founding principal and owner at Visual Data Group, a technology and business consultancy that specializes in data and information.

"Start small, build trust and then open that up on an almost absolute, rapid basis," he advises. "And you want to have data governance so that you understand what it is you're looking at. You can't improve what you can't measure." By acquiring, validating, building trust in, and then exploring data, organizations begin to build a system of metrics for assessing the quality and relevancy of data, he says.

It's also important not to let the buzzword “big data” distract process teams from what a big data-BPM effort really is: a project.

"You need good project management in whatever you're doing -- whether you call it big data or process management or process improvement," Harder says. "You still have to have defined goals. You still need some guidance, in an organization of any size."

About the Author

Stephanie Mann is the former assistant editor for ebizQ and its sister TechTarget site, SearchSOA. Before joining TechTarget, Stephanie was a contributing reporter and proofreader for a Boston-area weekly newspaper and an editorial intern at a Cambridge, Mass.-based publishing company. She has also worked for several nonprofits and as a freelance editor.

More by Stephanie Mann, Assistant Site Editor, ebizQ