We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.
Start a Discussion

What's the Best Way to Handle Unstructured and Ad-Hoc Processes With BPM?

Vote 0 Votes
As this has been quite a hot topic lately, what's the best way to handle unstructured and ad-hoc processes with BPM?

13 Replies

| Add a Reply
  • I noticed that recently a few vendors started offering Case Handling capabilities in their BPM products.

    For instance, IBM introduced "Dynamic Human Workflows" in its WebSphere Process Server, which allows run-time users to rearrange some details of the flow during its execution so that it evolves differently than the way it was defined at design time.

    I haven't had a chance yet to apply this kind of new capability to actual projects. So I would be very interested in getting feedback from people who have used those tools and could tell whether they helped at all in the practical execution of unstructured and ad-hoc processes.

  • This is certainly a hot topic and one that I am glad you are touching upon. And I am sure you are also aware that I just finished a 350 page book on this subject called Mastering the Unpredictable which should be available at Amazon any day now. The reason the title focuses on "unpredictable work" is because we are trying to avoid tying to any particular technology, but instead on what the true nature of the problem is.

    You question how to best handle with BPM, but that begs a more important question about the definition of BPM. We know unambiguously that BPM means Business PROCESS Management. All work is viewed as a process which can be defined and managed. In some very real sense, if you can't write down the process, then it isn't work as far as BPM is concerned. There are no BPM processes for executives -- so they don't actually work.

    When you ask about how to handle with BPM, many will hear the question as being about BPMS -- the system that supports the practice of BPM. Some believe that an existing BPMS can handle unpredictable processes by defining an empty process: a process without any structure to it. Maybe a collection of activities without any order. Technically you can do this, but it is not "BPM".

    The point is that the practice of BPM, based on scientific management, is at the very core fundamentally reliant upon having a process to analyze and continually improve. It also assumes a separation of brains and brawn: the people defining the process are not the people working the process. If your process is unpredictable and not repeatable, then you can not use a BPM approach.

    Instead, I believe a fundamentally different approach is needed -- a different method, a different practice. Instead of focusing on process, you need to focus on "business entities". Instead of a very thorough and complete BPMN diagram, you need a simple check list which people can extend and modify while they work.


    The biggest source of unpredictable processes is in knowledge work. I just make a slide cast on the subject of the nature of knowledge work.:


    So, the short answer is, you CAN'T handle unpredictable processes with the practice of BPM, you must use a different method to support this work.

    Can a BPMS support both BPM and a case management approach? I don't see any reason why a vendor couldn't add all the tools necessary to support both approaches, but that is not what you asked, is it?

  • The desire to "manage work" with BPM products must be yield some economic benefit. In order to work effectively, BPM products require a relatively detailed analysis of work (i.e. it must be predictable to some extent) and result in a relatively prescriptive model of work. The risk is that existing work practices are captured, modeled, and supported at a too fine level of granularity, making them brittle and prone to change. If the descriptions are too abstract, BPM products offer few benefits, as the work units to be managed are too abstract.

    There may be a class of work that seems unpredictable, but only because nobody has identified which events and decisions drive the flow of work. In this case applying a BPM paradigm might yield insight into the mechanics of the process, and BPMS can be deployed with success. But if work is truly unpredictable (e.g., if it depends to a large extent on events beyond the work system's control), taking a process perspective may not help much.

    There is a large body of research in the academic community on flexible and adaptive BPMS, which ease some of the limitations of earlier generations of process-support systems. In particular the Adept system developed at the University of Ulm is built on more than 10 years of research on flexible process structures. Similarly, the (discontinued) Wasa system at the University of Muenster allowed for runtime changes to predefined process structures.

    As Keith said, if it ain't workflow, don't try BPM.

  • This reply is a fragment from the following post “Let us architect the use of existing technologies instead of blaming them for bringing complexity/inflexibility/etc. in enterprises “– see http://improving-bpm-systems.blogspot.com/2010/04/let-us-architect-use-of-existing.html
    It describes the issue from four different views:
    • systems thinking view
    • BPM (as a discipline) view
    • case manager (or relevant authority) view
    • business system architecture view

    BPM (as a discipline) view is below:

    BPM as a management discipline (how to use processes to manage the enterprise) is about “to model, automate, execute, control, measure and optimise the flow of business activities that span the enterprise’s systems, employees, customers and partners within and beyond the enterprise boundaries?.

    The meaning of the word “model? has to be clarified. It means to make known, to describe or to communicate a plan (or plan of work or process template) of how to carry out future actions to obtain a desired outcome. Such a plan may have non-functional characteristics (risks, time, resources, values, etc.) and can be tested by some simulation. Usually, such a plan of work is prepared by the case manager and is subject to approval by some stakeholders of a particular case.

    The meaning of the word “automate? has to be clarified also. It means to instrument the proposed plan of work by some existing or new tools.

    Obviously, those 6 BPM functions (model, automate, execute, control, measure, and optimise) are applied iteratively and continuously.

    How often? It is a choice of the people who implement a BPM-centric system. BPM discipline does not limit them. Possible variations are:

    • 1 (to be precise, the same version) process template for N process instances – a traditional use of workflow technology.

    • 1 process template for 1 process instance – individual tailoring of a generic process template for a particular case. An example is “Defined Process? [capability level 3] from CMMI; another example is a legal procedure [bankruptcy procedure] defined by the law which is actually a skeleton (with a lot of exceptions) for all instances.

    • No process template before the process instance – the template is built as needed within the process instance (or at runtime). Small process fragments are used. Some of them are predefined in a library of process patterns, some of them have to be created on demand.

    What are optimisation criteria? Again, it is a choice of the organisation. The latter may use the BPM for perfecting internal work (inside-out) or serving each customer differently (outside-in).

    So, the BPM discipline is rather applicable for managing of cases.


  • I'm partly with Keith on this. BPM as a discipline is too structured to look at ad-hoc and unstructured processes. That said, I have witnessed a number of BPM/ BPMS tools which offer a way to try to manage these types of processes.

    The best way in my mind is to define the start and end points and figure out the big black box in the middle through adaptive process discovery. Every time there is an instance of the process kicked off it's tracked as it travels across the enterprise and touches the participants, it's not modelled up front and this is the key part. Once there have been enough instances tracked and discovered there is an element of probabilistic vs. deterministic branching to try to predict on previous behaviour where the work package or item will go next.

    Only when tools start to build on this notion and feed the information back into the system can they start to become predictive and pseudo-intelligent.

  • user-pic

    Think of BPMN diagrams translated to BPEL (or XLANG, etc.) as equivalent to the "structured" procedural code of COBOL programs. Where structured, repeatable processes with pre-defined and linked activities are procedural, adhoc and unstructured processes require the equivalent of event-driven GUI programming environments like VB or Java. BPM as a practice should track more than just activities and people, it should also track events, data objects and their state changes. With the help of event brokers and workflow systems, any work that occurs in unstructured processes can be tracked.

    To accomplish this we need to manage the life cycle of data objects rather than the sequence of activities in a process. To manage unstructured processes we need to timestamp state changes to data object instances, tie them to the events, people and activities that cause state changes to them. Managers can practice the discipline of BPM and process improvement using statistical methods on the logs of the workflow and event broker systems to trace data object instance life cycles. This way unstructured processes can be better understood in terms of the activities that are performed on those data objects. As an example, managers can attempt to minimize the cycle time of the data objects by analyzing the various sets of activity sequences that are performed on them.

  • user-pic

    I agree with the other commenters when they said that you must use the right tool for the job. It's easy to get caught up with a technique and treat it as though it will provide all the answers. BPM is not a panacea.

    It seems to me that processes are one of many possible views; similar to the views for different perspectives in the ISO/IEC 42010 standard for architectural description of software intensive systems.

    The challenge is knowing which views are useful but also knowing when to tightly specify and when to let go.

    Stafford Beer in the Viable System Model sets up a model, for want of a better word, that can help identify when the complexity becomes too great to comprehend. That is when the baton is passed into the next level to manage the details.

    Perhaps the problem is we're trying to maintain detailed models of all the processes at all levels (the Viable System Model is recursive so the same structure applies at several levels in the system of interest) but are drawing them to a single level. It might not matter at an enterprise level, what procedures are followed deep in the bowels of an operational unit as long as the critical performance, compliance and other outcomes are achieved.


  • It seems to me that missing from the debate is a discussion of a declarative approach. The pretzel twisting involved in trying to make today's imperative approaches flexible are, I believe, an indicator that a better approach is needed. I have no stake in declarative models; but they seem to me to be the answer to flexible workflows. It appears that right now they're confined to the cubicles of Eindhoven, but surely they must soon be ready to break out into commercial implementations.

  • I think it is impossible to give an answer to that question for two reasons.
    1. There is no universal agreement on a definition of BPM (both as a discipline or as a technology). For some people BPM is an application platform on which anything can be done, for others it has a narrower focus. Personally I agree with Keith and Michael.
    2. Ad-hoc, unstructured processes define a wide range of processes (and also means different things to different people) - from processes that can be handled by workflow+rules, to processes handled in email.

    So my glib answer is - as soon as people start using BPM instead of email for their ad-hoc, unstructured processes - we'll know that BPM suites have met the challenge of managing ad-hoc, unstructured processes.

  • What's the Best Way to Handle Unstructured and Ad-Hoc Processes With BPM?

    It's interesting that this seems, from the responses so far, to be such a tricky question to answer.

    It illustrates the way in which BPM has usually been defined [by IT and vendors] to focus on processes that are, or can be, automated. Everything else is brushed aside as unstructured and ad-hoc.

    But enterprises need to have a much bigger perspective. They need to understand, manage and improve their end-to-end processes. Everything, not just what's automated.

    Completely automated processes are very rare. Mostly it's a mix of manual and automated activities - and frequently it's mostly manual. Hence why BPM has had such little traction outside of IT.

    Automation is important but only part of the picture. And unless the enterprise has end-to-end visibility of its processes, its automation projects will be sub-optimal anyway.

    I see this holistic conception of process as the emerging business language http://bit.ly/cT0Ikx

  • The way BPM has evolved has been primarily focused on improving the productivity of the blue-collared part of the working population, that may include some knowledge workers too but again blue collared ones :) They are expected to follow what as been modeled by the business owners, and expected to accomplish their tasks from task lists in the order mentioned, and in a measured manner.

    A lot of professionals still go to work and decide for themselves as to what they do at work instead of a workflow or a well-defined model. They may participate in a strategy workshop, handle an escalated issue from a customer, take a sales call, make a series of notes from various meetings attended and so on.

    And there are a lot that fall in between - required to drive their own work while no one is driving it on day to day basis, but still not professionally experienced and mature enough to decide for themselves as to what takes priority.

    My initial reaction to anyone trying to separate an unstructured workflow or process from BPM is to object and reaffirm that it's also BPM but approached differently. However, I realize that in order to address the real "Work Management" that addresses the "truly" unstructured stuff, the current technology under the BPM umbrella is not sufficient.

    So, from this perspective I agree with the thoughts of Keith, Michael, and Jacob. However, I would look at this situation and say that this is an opportunity for us to really embrace the unstructured work as part of the BPM rather than separate it completely and create another silo'ed approach. I look at BPM Ecosystem as not just BPMS, but where the supporting technologies and core technologies come together to address the needs - for clarity represented here - http://ashishbhagwat.wordpress.com/2010/03/16/bpm-ecosystem-blurring-boundaries-or-systematic-convergence/

    A combination of BRE, (traditional) BPMS, CEP, and Case Management should be able to address this. And of course, there are tools emerging that are specialized in Case Management and Adaptive Process management. OR even what Theo proposes may work with predefined start ad end-points and adaptive workflows.

  • Many people believe that BPM is too structured to support ad-hoc processes and at one point in time, that was probably true. But today, companies should make sure they use a flexible BPM solution that will allow for changes as they learn and grow. Sooner or later, the need will arise to document workflow and introduce processes, and starting early with a flexible solution is the easiest way to accomplish this.

    Smaller businesses may not have a need to be rigid about processes. For example, if you only have five people in an office, you don't need a structured process flow, you just do what needs to get done. However, as you grow and orders get lost, people don't follow-through on tasks, and management doesn't have visibility into what's happening, that's when these "ad-hoc" processes must become more structured.

    I'd suggest that businesses start learning about BPM early and looking for a tool with flexibility and a low cost of entry. Doing so will force you to think through the details and workflow issues as you develop a solution with the tool. By learning as you go, ad-hoc processes reveal themselves and can be addressed before they become an area of weakness. In reality though, not everyone has the luxury of implementing BPM early. If you're in this position, flexibility is still the key. Make sure you're using a system that is scalable and easy to use.

  • user-pic

    First of all, it would be interesting to know the meaning of unpredictable and adhoc process… are we talking about the process where any activity can be executed in any order, or are we talking about process which is partially adhoc, i.e. there are some steps to be completed before adhoc steps are available for the user. In any case we will have to assume that at least list of possible steps is available to the users.
    Before discussing if a process which is adhoc can be managed or not, we are forgetting that managing a process using BPM not only ensures the order and predictability of tasks but also measures and monitors the process execution. If we can actually use Process Monitoring, one of the BPM concepts, in adhoc process with intention of monitoring how each process instance is executing, we may be able to find out most followed process pattern and even suggest the most optimized way of executing the process as well.

Add a Reply

Recently Commented On

Monthly Archives