BPM in the Real World
How Good Does a Process Definition Need to Be?
By Glenn Smith, Principal Consultant, Appian
Business process management (BPM) is often promoted as the key to achieving breakthrough
improvements in business performance. As companies move from functional to process-oriented
organizations, they need to align their technologies with the new approach. BPM
systems promise to deliver exactly that.
If BPM systems are so perfectly aligned with process-oriented organizations,
then why aren't all businesses using them? Why are many organizations that tried
BPM getting less than the expected results? Before answering these questions,
we need to digress a little and review exactly what a BPM-based application
At the core of any BPM application is a definition of the process. This definition
supports execution, control, monitoring, auditing and documentation of the business
process. The primary mechanism for this is through the definition of sequences
of tasks or activities. The system delivers each task to the correct resource(s)
in the correct sequence, along with any supporting documents or data. It records
the actions of users as they work on those activities for monitoring and auditing
purposes. It captures data entered during the process execution.
The earliest successful process applications involved high-volume, routine
processes such as insurance claims processing or loan application processing.
There is very little variation in the sequence of steps from one execution to
the next, allowing fairly simple system implementation. An explicit goal of
many of these was to reduce variation between instances. Many of these early
applications had the process embedded in the core application, and didn't use
an explicit process engine.
In the 1990s, general purpose workflow engines appeared on the market as platforms
for building any process-based application. Companies in many different industries
began applying these to many different problems with varying degrees of success.
One major obstacle in applying these to more complex problems is accounting
for all of the possible variations which need to be allowed for the process
to be successful. Adding all of the conditions and branching into the process
definitions made them difficult to develop, understand or maintain. Businesses
found it difficult to trust them due to the complexity. For some processes it
was simply impossible to model all necessary variations.
The next step forward was separating the decision logic from the process execution
by introducing rules engines. This allowed separation of the frequently changing
business rules from the process. The rules are represented in formats which
are easier for business users to understand and validate and easy to change
as the business evolves. Adding a rules engine to a process engine significantly
eases the handling of process complexity, but the fundamental problem still
remains. All possible decisions and outcomes must be identified and modeled.
Even with rules engines, it is impossible to achieve the business flexibility
inherent in human work.
Does this mean that we should give up and limit BPM application to rote tasks
with little or no variation? Of course not. The solution is to stop trying to
implement the perfect process and accept one which is good enough. The key is
to provide a seamless transition from the defined process world to the unstructured
human domain and back. Systems need to allow for ad-hoc deviation from the defined
processes and rules, but still capture the decisions and outcomes consistently
with the standard process. There needs to be a facility for modifying, during
process execution, the definition of the process for that execution only. It
needs to be done in a way that keeps all of the work within the process application
to support reporting and auditing.
This sounds contrary to a basic premise of process reengineering and BPM --
that the ideal process can be defined and enacted consistently. In today's competitive
environment, businesses need the flexibility to respond to rare or unanticipated
circumstances that could not be incorporated in the standard model. This approach
offers the best characteristics of both fully defined and controlled BPM and
completely ad-hoc human processes. Users are always hesitant to give up their
flexibility, and this is often an obstacle to implementation. During process
design, it is common to hear that it is not possible to enumerate all of the
possible ways a process could execute. With a flexible tool, it is possible
to model the great majority of cases while still allowing human discretion for
handling exceptional ones. This can remove one of the most persistent obstacles
to process acceptance.
The idea of runtime process definition modification has been described in the
research literature for at least 15 years, yet even today, surprisingly few
systems have implemented it. Platforms that offer this have a powerful advantage
in implementing the high value complex processes which defy complete definition.
One common approach to implementing this is to allow selected users to edit
a running process using some variant of the normal process definition user interface.
The running process is converted into a copy or new version of the process definition.
This definition can be modified and then re-activated as a running process,
with all history and state preserved. Of course the types of edits allowed should
be restricted to prevent fraud or audit inconsistency. The edits should not
be allowed to change the history of execution prior to the edit. For example,
deleting a completed task should never be permitted. If necessary though it
should be possible to re-execute previously completed tasks. A single process
can be paused, edited and restarted as many times as necessary. It is important
to note that when restarted, the modified process executes in exactly the same
manner as other processes, it just has a different process definition. This
ensures that work is delivered in exactly the same way, all audit trails are
preserved, and that summary reports of the business process can include modified
instances as well as standard ones.
A tool with runtime modification capabilities offers several advantages over
those with static process definitions. One benefit is reduced deployment time.
If users understand that they do not need to define every imaginable case, they
may be willing to move forward with a less than perfect process definition.
Another benefit of this approach is that it can be an effective tool for process
discovery. By examining the cases which required runtime modification it may
be possible to identify ways to improve the core process definition. If the
same type of change is being made frequently, then it should be incorporated
into the standard process definition. Over time, the defined process will handle
an increasing percentage of cases, reducing the need for runtime modification.
This is not to suggest that it is acceptable to be sloppy with process design
and let the users fix it by runtime modification. This is a powerful technique,
but one whose use should be restricted carefully. The fundamental premise of
process orientation remains unchanged. Businesses should analyze their processes
and document the best definition that they can determine. They should then enact
that process using an appropriate tool to execute, control and monitor it. Using
a system which supports runtime process modification allows the additional flexibility
to respond, within the system, to unanticipated situations without losing the
benefits of the system.
One of the distinguishing characteristics of today's most successful companies
is their ability to tailor their products and services to each customer. This
mass customization increases market share and also allows premium pricing. The
same principle applies to a company's internal business processes, since the
only reason to have these processes is to support the delivery of value to customers.
Business that can apply mass customization to their internal processes will
be better able to respond flexibly and creatively to their customers.
As noted above, most current products still do not support runtime process
definition modification. When selecting a BPM platform, businesses should be
aware of this, as it can have a large impact on the success of aligning your
company's IT infrastructure with its desired business model. Unless you are
planning to limit your applications to rigidly defined static processes, this
is a critical, but often overlooked feature.
About the Author
Glenn Smith is a Principal Consultant with Appian. He has more than 20 years of software development experience including more than 10 years implementing enterprise BPM solutions. He can be contacted at firstname.lastname@example.org.More by Glenn Smith