How Good Does a Process Definition Need to Be?

Untitled Document Business process management (BPM) is often promoted as the key to achieving breakthrough improvements in business performance. As companies move from functional to process-oriented organizations, they need to align their technologies with the new approach. BPM systems promise to deliver exactly that.

If BPM systems are so perfectly aligned with process-oriented organizations, then why aren't all businesses using them? Why are many organizations that tried BPM getting less than the expected results? Before answering these questions, we need to digress a little and review exactly what a BPM-based application does.

At the core of any BPM application is a definition of the process. This definition supports execution, control, monitoring, auditing and documentation of the business process. The primary mechanism for this is through the definition of sequences of tasks or activities. The system delivers each task to the correct resource(s) in the correct sequence, along with any supporting documents or data. It records the actions of users as they work on those activities for monitoring and auditing purposes. It captures data entered during the process execution.

The earliest successful process applications involved high-volume, routine processes such as insurance claims processing or loan application processing. There is very little variation in the sequence of steps from one execution to the next, allowing fairly simple system implementation. An explicit goal of many of these was to reduce variation between instances. Many of these early applications had the process embedded in the core application, and didn't use an explicit process engine.

In the 1990s, general purpose workflow engines appeared on the market as platforms for building any process-based application. Companies in many different industries began applying these to many different problems with varying degrees of success. One major obstacle in applying these to more complex problems is accounting for all of the possible variations which need to be allowed for the process to be successful. Adding all of the conditions and branching into the process definitions made them difficult to develop, understand or maintain. Businesses found it difficult to trust them due to the complexity. For some processes it was simply impossible to model all necessary variations.

The next step forward was separating the decision logic from the process execution by introducing rules engines. This allowed separation of the frequently changing business rules from the process. The rules are represented in formats which are easier for business users to understand and validate and easy to change as the business evolves. Adding a rules engine to a process engine significantly eases the handling of process complexity, but the fundamental problem still remains. All possible decisions and outcomes must be identified and modeled. Even with rules engines, it is impossible to achieve the business flexibility inherent in human work.

Does this mean that we should give up and limit BPM application to rote tasks with little or no variation? Of course not. The solution is to stop trying to implement the perfect process and accept one which is good enough. The key is to provide a seamless transition from the defined process world to the unstructured human domain and back. Systems need to allow for ad-hoc deviation from the defined processes and rules, but still capture the decisions and outcomes consistently with the standard process. There needs to be a facility for modifying, during process execution, the definition of the process for that execution only. It needs to be done in a way that keeps all of the work within the process application to support reporting and auditing.

This sounds contrary to a basic premise of process reengineering and BPM -- that the ideal process can be defined and enacted consistently. In today's competitive environment, businesses need the flexibility to respond to rare or unanticipated circumstances that could not be incorporated in the standard model. This approach offers the best characteristics of both fully defined and controlled BPM and completely ad-hoc human processes. Users are always hesitant to give up their flexibility, and this is often an obstacle to implementation. During process design, it is common to hear that it is not possible to enumerate all of the possible ways a process could execute. With a flexible tool, it is possible to model the great majority of cases while still allowing human discretion for handling exceptional ones. This can remove one of the most persistent obstacles to process acceptance.

The idea of runtime process definition modification has been described in the research literature for at least 15 years, yet even today, surprisingly few systems have implemented it. Platforms that offer this have a powerful advantage in implementing the high value complex processes which defy complete definition.

One common approach to implementing this is to allow selected users to edit a running process using some variant of the normal process definition user interface. The running process is converted into a copy or new version of the process definition. This definition can be modified and then re-activated as a running process, with all history and state preserved. Of course the types of edits allowed should be restricted to prevent fraud or audit inconsistency. The edits should not be allowed to change the history of execution prior to the edit. For example, deleting a completed task should never be permitted. If necessary though it should be possible to re-execute previously completed tasks. A single process can be paused, edited and restarted as many times as necessary. It is important to note that when restarted, the modified process executes in exactly the same manner as other processes, it just has a different process definition. This ensures that work is delivered in exactly the same way, all audit trails are preserved, and that summary reports of the business process can include modified instances as well as standard ones.

A tool with runtime modification capabilities offers several advantages over those with static process definitions. One benefit is reduced deployment time. If users understand that they do not need to define every imaginable case, they may be willing to move forward with a less than perfect process definition. Another benefit of this approach is that it can be an effective tool for process discovery. By examining the cases which required runtime modification it may be possible to identify ways to improve the core process definition. If the same type of change is being made frequently, then it should be incorporated into the standard process definition. Over time, the defined process will handle an increasing percentage of cases, reducing the need for runtime modification.

This is not to suggest that it is acceptable to be sloppy with process design and let the users fix it by runtime modification. This is a powerful technique, but one whose use should be restricted carefully. The fundamental premise of process orientation remains unchanged. Businesses should analyze their processes and document the best definition that they can determine. They should then enact that process using an appropriate tool to execute, control and monitor it. Using a system which supports runtime process modification allows the additional flexibility to respond, within the system, to unanticipated situations without losing the benefits of the system.

One of the distinguishing characteristics of today's most successful companies is their ability to tailor their products and services to each customer. This mass customization increases market share and also allows premium pricing. The same principle applies to a company's internal business processes, since the only reason to have these processes is to support the delivery of value to customers. Business that can apply mass customization to their internal processes will be better able to respond flexibly and creatively to their customers.

As noted above, most current products still do not support runtime process definition modification. When selecting a BPM platform, businesses should be aware of this, as it can have a large impact on the success of aligning your company's IT infrastructure with its desired business model. Unless you are planning to limit your applications to rigidly defined static processes, this is a critical, but often overlooked feature.

About the Author

Glenn Smith is a Principal Consultant with Appian. He has more than 20 years of software development experience including more than 10 years implementing enterprise BPM solutions. He can be contacted at

More by Glenn Smith