Business Ecology Initiative & Service-Oriented Solution

Michael Poulin

Why business process is always structured?

user-pic
Vote 0 Votes

Many people, especially those who deal with Adaptive Case Management (ACM), would not agree with the statement that any business process (as well as any process) is always structured. In my opinion, there is only one reason for this disagreement - understanding of what a process is.

From my discussions with several BPM professionals I've learnt that enthusiasts of process management believe that 'process is everything we do'. I suspect that this short formula hides one assumed element, which is the key to problems with ACM - this is the process logic or causes of our actions.

We are not simply doing, we understand something and we react to something by our actions. This logic differentiates an action from a process. But, wait a minute, do we know what/why we do things? Do we really know the logic of our actions?

Common knowledge tells us that we do not know what we do in many cases. We do not go crazy because of our own unstructured actions but and because we cannot even explain to ourselves the logic of these actions. If we agree with his, why are we obstinately trying to knot unstructured activities with a term 'process'? Expressions "unstructured process" and "unpredictable process" are oxymoron because we do not know what the logic of these processes is.

Nevertheless, we have locked ourselves in the "definition" of process - everything we do is a process - which seems inconsistent or simply incorrect. My understanding of the process is not 'anything we do' but only the things we can repeat by our 'doing'. If there is known logic, the actions may be repeated. If there is no known logic, there is no process and our actions may be unstructured.

Another reason to de-couple unstructured actions from process is in that we always expect particular outcome from given business process. If we do not know what activities are and how they collectively interact, we cannot expect particular outcome. This does not mean there is no outcome; this means it is unexpected and, probably, not repeatable. That is, next time when the same event triggers our activities, it is not necessary that we end up with the same result. I'd like to look at a business, which does not know what it would get in next moment. This contradicts to the entire idea of enterprise business model and business functionality/services.

Presented observation leads to simple conclusion - Adaptive Case Management and Business Process Management have only one common thing - business management. ACM is not about process management due to the absence of the process, it is rather about a management of consequences of unpredicted events, which itself is very important business task.

Chris Adams, VP of Product and Technology at Ultimus, believes that "the majority of processes in business have ALWAYS been unstructured". In Chris' classification, a workflow is the lowest hanging fruit, structured processes is the middle fruit, and unstructured processes is the highest fruit to attain. At the same time, Scott Francis argues that technically the unstructured processes are easier to implement than even a workflow.

Why we have such polar opinions? My answer is: because both use term 'process' inappropriately. Lotus Notes applications, email, or SharePoint are the examples of unstructured processes to Francis. For instance, what constitutes a process in e-mail? Well, if a process is everything we do, my question is silly. Doesn't this remind us a medieval alchemistic search for gold in everything? However, if we may assume that a process is about actions performed in logic, what are the actions and logic in e-mail?

Technically, e-mail is a means of transitioning of finite amount of information from point A to point B via mediating network according to concrete transport protocol. Sending amount of information from point A (if the network works fine) always results in the appearance of this information in point B. Between points, the information may be transmitted via different network passes but they are also finite and overall demonstrate quite structural behavior. If we add a human user into e-mail system, the number of human actions and related logic is very limited and may be enumerated relatively easy. This enumeration or possible combination of actions is very structured.

Finally, if we consider e-mail as an IT contribution/automation into the human process and argue that this very process is unstructured, does this constitute that e-mail is an unstructured process as well? No, it does not: we have no relationships but classis isolation of process - sub-process between e-mail and the mentioned human process. The last hope for unstructured process realisation with e-mail or any other interactive/collaboration tools is on the process of human dialogue. Indeed, we do not know whether the counterpart would respond to the sent e-mail but knowing the action and structuring the action are not the same things. In the case of a dialogue, interaction via e-mail is very structured: there may be only two possibilities - the e-mail will be responded or not (with some time-out variations). This gives the sending side enough information to structure its next logical step. If someone argue that our e-mail may trigger another e-mail to another recipient, I would say that this new e-mail belongs to another dialogue and out of the scope of our initial process; our process is fully structured.

If you give a pair of glasses to a monkey, you, probably, can observe unstructured actions. So, unstructured actions take place only when we do not know what to do in particular situation. This is why ADM has nothing to do with a process though they seem similar from an external observer. To deal with unknown, we need a mechanism for two things:

1) very flexible instrument that can easily adopt our actions for the case (and a process is the least flexible instrument because of its fixed internal logic);
2) an instrument that can quickly valuate the results of execution of particular actions.

The former is most likely a service because it abstracts process into function and results that are very easy to manipulate with while the latter is a rules engine or another engine that can apply our knowledge to particular fact - outcome of the action executions - and construct a meaningful recommendation for the further actions. The process can start at his point but not earlier, and apply its logic, i.e. structure of actions.

WebSiteAdvert.jpg

36 Comments

| Leave a comment

I agree with Michael.
Most cases, we standardize processes to increase quality and productivity, meaning that we make processes as routine and repeatable as possible.
Even limited number of highly complicated important exceptional cases, we do have expected inputs and outputs. Those are exceptional and important (corner cases), hence we do have certain business decisions and criteria, to make those work and a part of businesses.

Michael,
I disagree with (almost) everything. First off most BPM professionals don't believe that unstructured processes are actually processes - they are the hardest to convince that processes can be unstuctured. From my experience business people understand the concept of unstructured processes iummediately - BPM folks don't. BPM needs the structured provided by a model and if that doesn't exist it just doesn't fit their paradigm. Of course this is a generalization and there are BPM folks that grok unstructured processes - but they are the minority.

i don't think anyone would say email or Lotus Notes is a process. Those are tools - but they are often used to manage processes. Many business processes (e.g. sales) are actually managed that way.It is also true that email is used for many things that are not processes - not everything we do is a process.

I agree that the bottom line is the need to define business process - is it only structured activity - or can it include unstructured activity (with a framework\guideline\best practice and businesss goal). My definition of a business process would be: "A set of activities that begins with a mission objective and ends with achievement of the business objective". That covers both structured and unstructured processes - the structured ones have mostly system oriented activities, while unstructured, emergent processes consist mostly of human activities.

Jacob Ukelson - CTO ActionBase

Jacob, many thanks for the support: “off most BPM professionals don't believe that unstructured processes are actually processes� + “to define business process - is it only structured activity - or can it include unstructured activity�+� not everything we do is a process� – you confirmed ALL MY points!

You also say: “Of course this is a generalization and there are BPM folks that grok unstructured processes� – I do not believe that a generalization may contradict particular facts it had be inducted from - unstructured processes is oxymoron. With this regard, I would be very interested in learning why do you think that “business people understand the concept of unstructured processes immediately�? Is it because, again, ‘a process is everything we do’?
It seems that your definition of a business process – “A set of activities that begins with a mission objective and ends with achievement of the business objective� – fully confirms ‘a process is everything we do’, which I disagree with.

Based on my experience, people at the level of CTO and other C-executives use term ‘business process’ to identify (define the available functionality and its boundaries) the business capabilities - “A set of activities that begins with a mission objective and ends with achievement of the business objective� - what activities are, whether they are structured or not is immaterial.

I accept this semantic but note that this is exactly the definition of SOA service in Business.

So, I conclude that at the executive level in the Business (sorry, but I consider IT as a part of the business) business activities are described as business services.

In lower levels of business structure, ‘business process’ is always an _ordered_ sequence of business activities aiming business goals. No order, no process, but it does not mean that that there are no business activities; as you said: “not everything we do is a process�

Hi Michael,

Good post well worth discussion. I broadly agree. Since you are taking on the Case Management vendors let's start with Case -

A Case is a folder metaphor, a container for organizing artifacts (documents, forms, process fragments, etc.) and logging activities related to a business objective. Having a goal doesn’t make the Case a process, it makes it work for a purpose – a business service. Completing related, but disconnected, activities of a Case is not process. The inclusion of a process fragment in a Case, does not make the Case as a whole a process. The Case may involve multiple contributors, collaborative work, but that does not make it a process.

This is a semantic, not an existential question. That unstructured human activities are not processes does not diminish them in any way. Much of our work is performed this way, especially for knowledge-workers. I suspect that you and most of the readers aren’t actually subject personally to many business processes in your day to day activities, yet you work.

Now, regarding process let’s acknowledge that there is no process until someone models unstructured work giving it structure based on perceived patterns. In effect, process is an group agreement or a central directive to compress the rich variety of potential approaches that could have been taken to meet a goal, as well as reduce the range of potential events that could impact the delivery of service into manageable number of conditions.

The recording or capture of the ‘process’ from the unstructured ‘activities’ is a formalized abstraction with relatively low-fidelity. This is evidenced by the presence of ‘exceptions’ and the related need for ‘continuous’ business process improvement – both reactive efforts to adjust for the real-world. If we’re honest we’ll recognize that not every exception is reported. It's likely that many are recorded separately in shadow systems, on paper, or not recorded at all. Likewise, studies have shown that most continuous business process improvement programs tend to stagnate. The consequence results in operational inflexibility in practice.

So the work that remains unstructured activities may lack formalism for standardization, repeatability, and efficiency of modeled process, but it provides greater flexibility of execution that might just better align with circumstances and allows for creativity/ingenuity on the part of the responsible staff for greater effectiveness in meeting goals. Business should have both capabilities.

For our part, we've taken a hybrid approach where our technology evaluates every interaction to determine amount of structure required procedural when necessary, flexible when possible.

Our approach relates to your 2nd Mechanism for managing the unknown - a form of dynamic dependency injection. Activities are modeled, not processes. The process emerges from the activities. This keeps the focus on the work and the goals, as opposed to a fixed sequence.

Regards,
Dave

A brilliant post, Dave, thanks a lot.

Talking about your hybrid approach, do you have a methodology for this? How you measure the "amount of structure required"? Do you use any forms of rules of simulations (probability prognosis)? Do you have a catalogue of existing processes that might be engaged for appropriate case? I think that your experience should be well met in the industry.

Hi Michael,

I think one of the major problems of "BPM" is that it has been far too much identified with "control flow based workflow schemata". This happened to such an extent, that even a fundamental term like "business process" is perceived as such a workflow schema thing: something with a start, an end, activities inbetween connected by control flow.

This is utterly wrong!

If we look back at pre-BPM times a process then was simply something like a set of changes, of transformations. Think of chemical processes, modernization processes, customer acquisition processes etc. Applied to work, these changes become "intentional changes", i.e. changes performed to achieve goals.

In these terms, a process does not have a defined start and end, since it's just not a connected sequence of actions! Of course the single work thread (workflow) has start and end, but the process may continue forever. Compare:

a) a single acquisition of a single, specific customer (a work thread)
b) the acquisition process, consisting of all activities related to acquisition, lasting as long as the company exists
c) a schema, describing many instances of (a) and/or alternatively or additionally describing, but not being (b)

The distinction becomes more clear if you think of the "acquistion process north" and the "acquisition process south": same schema, but different change sets!

Based on these obervations and a few more crisp definitions we formed a modelling language, the UBPML, which is capable of expressing schematic processes, single projects or cases in a consistent manner. UBPML stands for "Unified Business Process Modelling Language" and is published under a creative commons license in a wiki open to the public (see ubpml.org).

In these terms, there is no contradiction between BPM and ACM. The underlying structure of work is no longer a controlflow web, but a hierarchy of goals. Such a hierarchy exists everywhere where work is executed. Certainly, in routine work, we're not always aware of it, but in such cases collaboration is just as little possible, if it's not already routine, too.

We call the thing that connects a given situation with a goal a "Step". Now, a case is nothing else but an instance of a Step. A project is a set of connected steps (connected by goals, milestones etc., not by controlflow). In constrast, a Step class categorizes similar Steps, just as classes always do. A set of connected classes describes a workflow, and a set of possibly unconnected classes might be used to describe a process. Even more: processes, workflows or projects described this way need not at all be something complete and big - to the contrary, since there's no such requirement like to interconnect everything with controlflow, rather incomplete single cases can be described consistently and uniformly with larger projects or even schematic processes.

In our wiki we present a simple meta model made from these terms which can be used to build process aware software. Actually we did that ourselves. We invite everyone to participate in our initiative.

Andreas

Andreas, I am very glad that this publication has caught your attention and pulled so many different viewpoints.

You may call me out-fashioned but I count the term ‘business process’ from early publications about manufacturing production as well as from Value Chain concept. To me, business process is very concrete thing which I do not mix with chemical or bio-medical processes.

I agree with you that a business process does not include the beginning, i.e. the trigger, but it always has the first step and the last step. BPM to me has nothing (OK, almost nothing) to do with SW automation but rather with management of ordered business activities (to protect them from deviations). As of your examples, I, probably, have not got them yet but (a) process of an acquisition of a customer starts to me with the first documented business decision targeting such acquisition and ends when the customer positively responses to your initiatives; everything else what happens with your relationships with this customer belongs to another business process, which can be performed by absolutely different business team. I disagree with your (b) and (c) examples – see my explanations to the first case.

To me, if the "acquisition process north" and the "acquisition process south" have exactly the same business logic, they are the same process applied to different original materials – north and south.
Nonetheless, I am quite interested in the "Unified Business Process Modelling Language" (aka UMLfor objects?)

I do not think that there is a contradiction between BPM and ACM because the former is about structured/schematic business activities while the letter is about unstructured business activities. We cannot describe unstructured activities with a scheme because these activities are performed with no particular scheme/structure in mind. When the activities complete, we can retrospectively construct a process for this particular case (as Keith Swenson described in his post) but it might be a one-time-and-never-repeated process which very questionable value.

Indeed, “there's no such requirement like to interconnect everything with controlflow� but if there is such requirement in particular case, we call it a business process. In other cases, we should call them differently to avoid confusions. Would you agree?

Thanks Michael.

We do have a methodology, a sole canonical protocol that’s at the heart of our patent pending system (www.ideate.com).

Our protocol enforces transaction controls, governance policies, and business rules, as well as the construction of the system response. In Ideate, every interaction is evaluated at run-time and the interaction context (in-band and out-of-band) is matched with applicable rules (run-time contract modeling / dynamic dependency injection), resulting in a custom system response with the valid next steps from that state. There are no pre-defined sequences and no hardcoded pages/responses. Each Case has its own lifecycle as befits its circumstances.

In general, the system response for a set of like circumstances can be predicted and is repeatable as it is based on logical rules defined by the business application owner(s) – they can be simulated.

The degree of structure is dependent on the rules defined by the business owners as appropriate for the situation. Unlike conventional BPMx, structure and sequence is not presumed, it’s a calculated response. The Method makes structure a variable within and across Cases. Some circumstances might require more structure for compliance, a series of specific steps with little latitude for the user to influence (ala BPMx), and some circumstances might allow for open user self-directed work/collaboration (ala Case Management) - this is the apex of lean and agile.

In this way, by your definition, we do support a process, an emergent process that unfolds step-by-step. We use a Case metaphor to organize the related artifacts (participants, documents, linked Cases, etc.), but ‘Actions’ are system mediated, and Case status is system controlled.

The System features a meta-programming layer. Every interaction requests one of just a dozen underspecified reflective programs. These programs are executed by a service acting as an agent, which resolves abstract references by walking link relations in a virtual information repository based on metadata.

The virtual information repository stores everything (data, business entities, rules, functions, services, program code, attachments, etc.) as a resource. The system maintains the lifecycle of every resource, not just the Cases. Every resource has security, history and roll back control.

The meta-programming layer adds generalism to the entire system and facilitates re-use. Developers can re-use existing Resources and/or applications to derive new Resources and Applications. The system comes with Resources for rich forms, robust budgeting/financials, and a wide variety of review mechanism (including agendas, board meetings, minutes, peer reviews, etc.). Expert-system type applications can be developed in 2-6 weeks.

Unlike SOA and BPM deployments, information is a first-class citizen, which allows an expanded definition of context based on all Case content and nested relations. Our Method goes beyond simple building block approaches where states are dynamically chained based on limited conditions (‘data driven’) with fixed responses. In addition, the Method does not have the practical limitations of calculating context against a Finite State Machine; custom constructed responses are based on recursive refinement of a sub-graph.

In short, our Framework is capable of just about everything Pegasystems’ Object Oriented rules driven process management system can do with less middleware, latency, and expense in a smaller footprint with greater adaptability.

Sorry for running long, but I wanted to be responsive and build on our past exchanges.

Dave

(@Dave: I read your comments with interest, too)

Michael,

thanks for your response to my comment, I very much appreciate discussing these different viewpoints.

Concerning terms, I certainly would not call anyone out-fashioned, the shift of meaning of "process" happened world-wide in the community, no-one's to blame.

And yes, of course I agree that "process" is used at present as you describe it. All I want to say is that that usage is unfortunate, since it leads to the perception of "BPM and ACM do not (really) fit", which is wrong. My distinction a), b) and c) describes a more usfuel classification, but I'm quite aware that it is not the usual one.

The matter of specific terms aside, fact is that a comprehensive approach to BPM _must_ address all facets of the topic "work", which comprises pre-planned schematic workflows as well as the more loose versions: not-preplanned, creative, dynamic, cases, projects up to, yes, modernization efforts. We should speak about that in precise terms - at present ours are not really good.

BPM as structured and ACM as unstructured is only a first step in seeking better understanding. It divides the world into two parts, which is ok for purposes of understanding, of analysis, yet in day-to-day work, that distinction does not exist. In all situations I have seen there is never ever pure pre-planned or pure unstructured work, it's always a mix, nothing is completely programmable, nothing is completely unpredictable. Even the term "unstrcutured" is wrong, there is always a very concrete structure, which is the goals we want to achieve collaboratively. "Unstructured" in fact is used to describe situations where we have no pre-existing controlflow schema classifying our work thread in advance. "Unpredictable" refers to the fact that the specific solution procedure is not known; the outcome very often is predictable. Of course there is also work with unpredictable outcome, but that's not the majority. We have to be careful here to not throw out the baby with the bath water.

Moreover, if we try to achive goals and break that achievement down into subgoals which are collaboratively achieved (nothing else is an enterprise), the schematic preplanned approach and the open and creative version ("BPM vs. ACM") might occur in an intermangled way repaetedly; even in open work situations subgoals might occur which are in turn solved by precisely preplanned subprocedures (business and technical services), and vice versa.

Finally, there is another reason why I would like to (re-)recoin the term "process" again away from "traditional controlflow based schemata". That reason is simply the business side of the whole thing. BPM, to me, is the "art of managing processes", as it is used in pure IT-free business and management thinking. And managing business processes in that demain for sure comprises all types of work, no doubt. So, if we reserve "process" for that controlflow variant, how should we call the management discipline?

Andreas

First, the answer to your last question, Andreas:
this 'thing' may be called something like Business Activity Management.

I think, it exists already but frequently under wrong name of BPM. Then, I do not see a need to merge structured and unstructured organisations additionally to the merge in the enterprise.

Back to >>"Unstructured" in fact is used to describe situations where we have no pre-existing controlflow schema classifying our work thread in advance. "Unpredictable" refers to the fact that the specific solution procedure is not known

I fully agree with you in that process and unstructured activities appear in "an intermangled way repeatedly" but we need a solution for unstructured activities as we have such for the structured ones. You do not deal with carp and shark the same way though both of them are just fishes, correct?

Dave, your last response is very important (to me, at least) because it describes your solution specifics in very comprehensive way.

When I read it, I recognised some elements of IBM's Dynamic Process Edition (a process loosely coupled with actions that are preliminary agreed with potential action providers via Service Contracts but engaged from a repository of actions on concurrent or alternative grounds based on the process' decisions).

By the end of reading, I understood that your solution is an implementation of my definition of process-service and unpredictable conversational model of XML message exchange that I described in one of my articles in Sys-con and in my book about 2 years ago. Particularly, process-service is a process realised via service orchestration where everything including process business logic is provided by a set of services. The actions in this case are the services in the IBM's Dynamic Process meaning. This provides logical abstraction for each process where you can ignore actual implementation for the sake of solution flexibility.

Also, for the unpredictable conversational model, a set of XML Schemas may be created as a vocabulary and the consumer and service exchange messages using this vocabulary. Consumer may require responses in certain Schemas (data structures and and processing commands) while the service does not know up-front what the next request would need. You can extend or squeeze the vocabulary without affecting those consumers who are not interested in added/removed schemas - all going through the same service interfaces with no changes in the operation definitions.

I am happy I we have an acquaintance now; I will be in touch directly, if you do not mind.

> ...BAM...

Maybe that's a good choice, if we want to reserve the meaning of process to controlflow schemata, I wouldn't prefer that. Anyway - most important to me is consistency instead of confusion. But that's only partially in our hands :)

> ...we need a solution for unstructured activities as we have such for the structured ones. You do not deal with carp and shark the same way though both of them are just fishes, correct?

Certainly. From an architectural and oeconomical point of view I'd add that there are still many similarities (both are fish ;-) ) and we should try to avoid redundancies, reinventions and so on as much as possible.

E.g.,
- an employee works on a task and wants to complete it - much likely he wants to work on it or complete it independent of how it was created (by BPM schema or by manual ACM instantiation).
- one might want to count certain incidents each month independent of where they originated from.
- in an ACM work scenario one might want to start a predefined subsequence.
- one might wan to treat process, project (case...) with the same tools.
- actually, in day-to-day work, one might not even want to think about whether something classifies as "BPM or ACM", it'd be an artificial burdon

There ARE of course huge differences in culture, habit and style of the organisation, but our methods and tools should just be flexible, not predetermining.

Yes, Andreas, I agree "we should try to avoid redundancies, reinventions and so on as much as possible."

It seems that if we are not ready to particular event - we do not have reserved process or we do not have an idea what to do with it - we engage ACM, which eventually ends with BPM. In the BPM, we also have a place for ACM (see below) and, again, may end-up with BPM, an so on. Thus, they work together an address two sides of the same event depending on our readiness to face it.

Now, back to ACM 'inside' BPM: actually, when it happens in BPM that the only manner of actions available is ACM, the BPM stops there, i.e. ACM is alway outside of BPM. In BPM, there may be a situation when the operational control delegated to the action and never returned. If BPM does not apply a 'time-out' or cannot move forward without the action response, the process has to stop... or pause. The control may be not returned because of many reasons like communication line brake or unexpected latency in the action performance ( a request came to the Bank but the manager who has to approve the response is taken to emergency and stays in the hospital). So, unstructured activities must be performed to restore the flow of information and actions that may end-up with returning to the original process (BPM) to the point where it paused.

I am cheating here a bit - as I said before, the process does not care how its actions are performed, i.e. are they processes or unstructured activities. That is, we have to have a mechanism for switching from structural to non-structural activities. This mechanism is the answer to BPM & ACM separation and collaboration, IMO.

Michael, 287% agreed!

In my opinion, there is no way to establish such a mechanism if we continue to use controlflow as we did before.

Before explaining, let me define two terms: G-process (general, i.e. just a set of transformation instances) and CF-process (a control flow based activitiy schema); not for the sake of bossiness, but because I need something to denote the first and to distinguish both.

Now, the idea is roughly: not to use CF-processes anymore to describe all kinds of pre-plannable G-processes. Instead: rely on results to connect the various steps. More precisely, a result is a set of artefacts in a certain state. We're already very used to this type of connection: in project management! Here, we have milestones, deliverables, results we use to communicate state and to collaborate. It would not make any sense to say e.g. in a project "Yes, the concept task is completed", without a concept document to present. But in CF-processes we do exactly that: we say "Here's our token" without looking at what happened. That _does_ make sense in certain specific situations, where we can be 100% sure that the results are there if only the token arrives. E.g., in computer programming we do that regularly, a computer program is nothing else but a sequence of actions performed "blindly" (of course there are assertions, pre- and postconditions, invariants, but that gets us off the track).

In UBPML we use the term "constellation" to denote such a set of artefacts in a certain state. So, let me call the respective descriptions C-processes (C: constellation). Such constellations could be e.g. "there is an unprocessed customer request", "stamps ran out", or "there is an unconfirmed request". C-processes have interesting properties: they provide a structure to pre-plan work or to develop solution pathes, as CF-processes do. In addition, they provide exactly the required "check-out" property to "switch" to ACM. Even better: they eliminate the need to switch, since C-process-tasks are always "out of control flow". They are completed if and when they are completed, whether that was performed by a machine, preplanned or freely and by creative means. In fact, with this approach there's no difference between the two approaches. Therefore also exceptional situations are no problem any longer. If a task does not complete because the manager threw his computer out of the window - no problem, the "unconfirmed request" stays simply unconfirmed, everyone can see that and solve it with whatever means he likes. The request stays in the system as unconfirmed, there's no black token hiding in the task of that manager.

With the distinction between classes and instances (constellation class: "some request arrives", constellation instance: "request #4711 arrived") and with object oriented mechanisms C-processes provide further means to organise processes flexible and comprehensible.

There are two interesting properties to observe: first, if we define a constellation as "the previous task was completed", it becomes obvious that CF-processes are just a special case of C-processes, and the problem of controlflow becomes equally obvious: it abstracts from too much details, see project management example above. And second, there is a strong relation to event processing: what are typical events? In business scenarios almost always "constellation X was reached", here we are.

Andreas

Hi, Michael,

I belong, as you know, to the ACM/BPM community and I believe that all work is in pursuit of building and improving customer experiences.

This means the outcomes of the services and/or goods provided have to be predictable and consistent. A construction company tries to avoid cost/time overruns, a manufacturer of a new model of camera wants the new model to work at least as well as the previous model. Deviate from this and you lose customers.

Now, in any workflow execution environment, I need to accommodate tasks.

Each task is somehow in pursuit of the goal otherwise we would not require that it be done. Some tasks contribute directly to building and improving customer experiences, others contribute indirectly. And BPI presumably eliminates tasks that do neither.

A cluster of tasks can be included and interconnected in a BPM workflow so that those who believe in and like to use ‘best practices’ can do so.

No workflow can cover all eventualities in a complex situation – so the capability to perform tasks out of sequence, skip tasks, re-visit already committed tasks and perform new tasks that for whatever reason were not in the workflow leads to a need for an ad hoc task capability.

So as not to make a special case of ad hoc tasks we have workflows with multiple tasks and then we have ad hoc tasks that are each workflows of one. So everything is a task that is a member of a process. You can have 10,000 processes if need be

The basic concept is the same as a menu at a restaurant. Order a meal or go a la carte.

Clearly if I manage manually to perform the same series of tasks in the same order as the best practice then I will reasonably get the same result. But, not everyone plays the game this way.

This brings us to making a menu of ‘pickable tasks’ that basically include all of the individual tasks you find in the organization’s workflows plus all of your structured workflows and letting the user pick one menu item and then another. Or pick several at once.

The brick wall comes when a user does not follow a BPM workflow and performs different tasks in a different sequence from the best practice. We have two choices – disallow ACM or allow it.

I allow it because the environment I use has resident background independent compliance control such that whether you get to a compliance checkpoint task along a BPM workflow or whether you pick an ad hoc ACM task, as and when in your processing you reach a ‘key process point’, compliance controls kick in and you will be barred from going forward if deficiencies are noted. Remedy the deficiency and you can revert back to processing.

Example: You cannot in a hospital do any task related to ‘discharge’ until the patient’s physician signs a release form. Each workflow, each ad hoc step that has anything to do with ‘discharge’ will have a rule that looks back to see if we have a signed release form and if not, then things grind to a halt.

For me, it’s unreasonable to say everything is ad hoc or that everything is structured. Sometimes its 80/20, other times its 20/80.


Karl,

I think your expressed in your words the idea I tried to articulate in this thread earlier: when we are getting off the obvious next step, we can engage a set of rules hoping that there will be a way to decide what the next step will be. If the rule engine fails to produce meaningful decision, we are going with ACM. And this is good.

The unstructured actions are the actions where decision about next step brings the step that we were not able to formalise before and, in the is no guarantees that in future the same 'next step' will be decided for the similar situation. So, there are three, at least, causes for ACM: 1) we do not have a prepared process for particular situation but we anticipated it; 2) we cannot recognise the situation and do not know what to do but elevate the issue; 3) we have case where the instruction tels us to elevate the issue and we do not know what will come back from there.

In the case of use of rules, we certainly getting into the menu-like model. I am with you in this. However, ACM to me is not about ad-hock actions but about the actions and decision points that return non-planned/anticipated next action. I suspect that one of the reasons for such behaviour is in that processes/BPM traditionally focused on their own tasks and ignored Execution Context, which can influence the decision about the next step and bring 'unexpected' actions.

Michael,

Just to be sure, in your title does "..business process.." refer to process-template or process-instance? The former is structured by definition, the latter -- all depends (see http://improving-bpm-systems.blogspot.com/2010/12/illustrations-for-bpm-acm-case.html)

Thanks,
AS

Thank you, Alexander (AS), I quite agree with the first comment made by Dave Duggal to your post - "nicely distinguish procedural process from emergent process".

Actually, I thought about similar model and even started wring a new blog but you did it first.

I am trying to comprehend the difference between the 'process-template' and 'process-instance' in your vocabulary. So far, I've got that the process-template is a set of ordered actions in accordance with preliminary defined business logic, i.e. it is a business process in my words. The process-instance is an attempt to identify actions in the absence of preliminary defined business logic, i.e. "emergent process" in Dave Dugg's words though we cannot be sure that it is the process that is emerging.

I accept process-instance as a half-step toward understanding that: if there is no preliminary defined business logic, then there is no business process but rather a set of unstructured/sporadic/ad-hock business actions.

Thus, my title refers to the process-template in your terms that, in my opinion, is equal to actual real business Process.

I deeply disagree with the statement: everything what we do is a process. To me, everything what we do is a set of actions. In the next step of classification we find if there is a structure - it is a process, if not - it is still a a set of actions that demands a special case management.

Michael -

Interesting read. I'd point out that when I listed tools (sharepoint, notes, email) - I was listing the technical underpinnings that people used to manage "process" which either was fundamentally unstructured or hadn't yet been understood, analyzed, formalized, and structured. Often even an ordered/structured process is still implemented using these tools (or paper).

You ask what constitutes a process in email? Getting approval for funding a marketing event, or sponsoring such an event, is one example. It could obviously be a structured process - but in many organizations it isn't. At the start, you may not know who needs to approve the expense - you may not even be in marketing. But you can use email to communicate with co-workers and find out who it is, and to get a paper trail of approvals if needed. Even if the process is well-defined - and everyone knows the process - it may still happen via email communications. There are thousands of other examples.

Incidentally, I'm fine with my own use of the word process :)

Scott, thank you very much for responding. Yes, I assumed you mentioned e-mail as a technical means that might be used in unstructured activities (I cannot say 'unstructured process' - it is an oxymoron to me).

I do agree with you that e-mail may be used for communication. However, the example you described in your response. If you do "not know who needs to approve the expense", who you send your e-mail to and what constitute your process to claim that your next e-mail or an e-mail of arbitrary receiver in in the financial team belongs to your initial process? Is is because a stack of messages or because of the same 'Subject'? What if a message about your case is combined by somebody in finance with another information unrelated to your case - is it still your unstructured process?

I do not mean to convert onto my position you but I cannot comprehend and accept the logic of e-mail being an unstructured process. Even if you use e-mail (which itself is absolutely structured) in an unstructured actions, this is not enough (to me) to justify that the e-mail is unstructured.

Michael,

Process-template is a plan (an abstract description of the process [which is an explicitly-defined coordination of services to create a particular result]) and process-instance is enactment of a process template.

In general, I found your point of view a bit “black-and-white� although in the reality _all_ options can co-exist within the same process(-instance) – see the note 2 from my post.

Thanks,
AS

Alexander,
I have posted another BLOG "Trying the Next Step When Doing an Adaptable Case Management", where 'black' is mixed with 'white' but... NOT in the process.

I find a comfort in distinguishing structured activities as Process and unstructured activities as a Case of activities.

I see that even you differentiate between a Process-template is a plan and reality, where structured is mixed unstructured. You call it 'instance of process' while I do not recognise such mixture as a process.

To my knowledge, business Process (in real life and in technology) does not require prior knowledge about particular services but only about the 'explicitly-defined coordination' of needed action results.

Hi Michael,

I didn’t realize this thread was still going until I saw your latest post. It’s great to see the exchange draw so many folks from BPM, SOA and ACM spheres.

Drawing a clear conceptual and technical distinction is good. If anything, I think a failing of ACM has been the inability of the community to succinctly distill it’s technical and business differentiation. That being said, Keith Swenson recently had a good thread in the ACM LinkedIn group re: defining the Business Use Case for ACM.

I imagine your post is causing some separation anxiety as you are suggesting that conventional Case Management be stripped of a claim to ‘unstructured’ process. Many if not most of the vendors embed some notion of process fragments into their case, and see the Case as the glue between fragments and part of the continuum, rather than simply work between fragments. Either way you see it, Case Management-type solutions still provides value by supporting unstructured actions to enable greater involvement of human discretion and events.

Regarding the need to “have a mechanism for switching from structural to non-structural activities. This mechanism is the answer to BPM & ACM separation and collaboration, IMO.�

We can do both, in one system – we make degree of structure an interaction variable based on rules. This distinguished us from ACM as it is being defined and we don’t need BPM though we can interoperate with it. Our system does the 'switching', it is treated as a logical continuum. As per Alexander Samarin’s statement “all options can co-exist�.

The system is always anticipating a next step, it’s interaction driven with ‘event messaging’. Since our meta-programming layer features underspecified reflective programs our system achieves a non-deterministic response based on a deterministic method (i.e. a consistent method for managing variety).

Part 1 of 2

2 of 2

Sorry for running long, but wanted to catch up on the thread and our sidebar discussion re: dynamic configuration -

I think your statement “To my knowledge, business Process (in real life and in technology) does not require prior knowledge about particular services but only about the 'explicitly-defined coordination' of needed action results.� is spot on.

Like you we make no bones about system mediation, our system coordinates loosely-coupled independently evolvable resources in executing emergent process. The system manages case state (it’s not user self-selected) based on actions and builds responses (including next possible actions) based on context. Our cases are not loosely connected artifacts collected in a container, they are highly-coordinated resources that exhibit collective behaviors based on link-relations.

Thanks for sharing your approach to unpredictability. We both seek to add flexibility to interactions, but we have clearly have very different system approaches (architecture, artifacts, methods, etc.)

We have a distributed Resource-Oriented architecture that is specifically designed as an enterprise information system, as opposed to a message-based distributed interoperability system. In the Ideate Framework, all system resources reside in a single virtual layer – data, program code, rules, services, etc. They may be distributed, but they are not in different components, they are addressed directly.

Our system achieves code locality, mobility, and caching to optimize the real-time coordination of resources in executing dynamic contracts. The directness of our system allows us to make a dozen or dozens of recursive calls to resources to evaluate interaction context and construct a system response - sub-second. We filed our patents two years ago and they are now backed up by implementations.

I understand that in your approach you “…ignore actual implementation for the sake of solution flexibility…�. This means your WSDLs are based on some form of ‘shared understanding’ of schemas. This would allow for some generality in your interface, but would tend to induce implicit coupling that could cause a governance headache or at least considerable workarounds,
Even if that is addressed, it would still leave the latency challenge of making dozens of ‘expensive’ calls to services. In the end, it defines a form of pipe-and-filter SOA with limited and/or expensive dynamic configuration of services.

Your conclusion in one response - “I suspect that one of the reasons for such behavior is in that processes/BPM traditionally focused on their own tasks and ignored Execution Context, which can influence the decision about the next step and bring 'unexpected' actions.� – sums it up perfectly. BPM deals with process variables, it has no profound connection to Case or Process Content, which is generally distributed across one or more relational databases. For data BPM depends on SOA or some other form of integration, but that has practical limitations as described above. There is nothing about a SOA/BPM deployment that makes this holistic. The cost of loose coupling in SOA is low cohesion. It lacks unifying principles to make context-enhanced processes practical.

Dave you 'force' me to respond. There are a few points with no particular priority.

1. There is a difference between definitions and relationships on one side and vendors' implementation on another side. I look at BPM/ACM as at a sort of a State Machine: in one moment the business activities in one state while in next moment the state may change to another one. If both states are delivered in one tool does not change the essence of the states.

2. I do not think that Case Management is a glue because many processes can communicate in chain with no unstructured intersections, i.e. w/o Cases. I am not sure I agree with you in that "Case Management be stripped of a claim to ‘unstructured’ process" - on the contrary, I suppose that the Case Management takes place exactly when we do unstructured actions (this is why it is called 'Case', i.e. special)

3. I do not see any discrepancies between my position and Alexander's one - I do not separate BP from AC by a 'Chinese Wall'; moreover, one of them triggers another one in no particular order. This means that “all options can co-exist� but only one of them runs in particular moment.

4. I would not say "The system is always anticipating a next step" about the system that listens to an event. Somehow, in he context of this discussion, an expression 'next step' associates to me with existence of previous steps while new event may cause a chain of actions with no relationship to previous steps. Thus, if you say that the system is always ready for next cycle of processing, I do agree with you.

5. You say: "Our cases are not loosely connected artifacts collected in a container, they are highly-coordinated resources that exhibit collective behaviors based on link-relations." - To me, it is not a simple statement. If you mean that coordination is an on the fly chain of decisions and these decisions drive the actions, I can take 'these decisions' as coordination while the actions may be fully decoupled artifacts. If used decision logic is consistent, than the overall Case behavior is collective; otherwise it is accidental ( a decision maker elevates the decision to the upper layer where previous steps in the Case are unknown and the case receives non-integral decision for the next action)

6. The major difference between yours and my approach, IMO, is in that yours is a resource/data centric while mine is function/service centric.

7. About "‘shared understanding’ of schemas" - we all including our products and actions are coupled by the informational realm. If we do not understand each other, we build a Babylon Tower. In SOA and any business functionality (as well as in technical one) the problem is not in coupling by semantics but opposite - decoupling because of inability or ignorance to preserve common/shared semantics. Note, technical ability to do translation worth nothing if there is no intent and willingness in the parties to interact, and the absence of these two qualities is the direct result of non-shared semantics: consumer does not understand what service provider offers or understand incorrectly - all this happens _before_ any physical interactions (in Business and in Technology). OASIS SOA standards contribute to this matter a great deal of content.

I do not really know what you mean by "dynamic configuration of services" but you have to remember that in SOA each service may have its own ownership and authority, i.e. IT may be not in position (any more) to tune services to work in any other way than the service owner and provider has decided. If the service is expensive, the consumers will vote by their feet to a cheaper services.

8 Finally, SO Principles include (see my blog 'SO Principles reviewed')a Principle of Service Execution Context while Principle of Loose-Coupling does not affect cohesion at all. Remember, SOA deals with distributed solutions for flexibility in adopting business changes. At the same time, it is well known that a monolithic application may be the most optimal from optimisation perspectives but... it is inflexible. Yes, SOA pays by performances for the capability to adopt the change first and be in the market first (get more revenue while others still modify their applications).

Michael and Dave,

please apologize for jumping into your discussion...

@1. ... BPM/ACM ... a sort of a State Machine...

Yes, true. If you structure "global system state", you end up with "constellations", i.e. sets of artefacts in a certain state.

- In EPC that was called "Event",
- in BPMN it was almost dropped and reduced to "A token arrives", optionally annotated with "Artefacts",
- in (A)CM it's not exactly modeled, but kept and stored in one place, the "Case Folder",
- in CEP it also corresponds to "events" (which add an (unneccessary!) timely component: "happening now"),
- with BRs it's coded into conditions,
- finally, in UBPML we name it "Constellation".

@2. ...processes can communicate in chain with no unstructured intersections, i.e. w/o Cases...Case Management
takes place exactly when we do unstructured actions...

I'd distinguish these aspects:

a) completeness (fully preplanned) vs. openness (unforeseeable)
b) schema (or template, class) vs. instance
c) interface (goals, steps) vs. implementation (procedures)
d) token-based coupling, goal-based coupling, arbitrary coupling

In controlflow based BPMS there's only "complete token-based coupled schema with implementation", only at runtime there are unchangeable instances, and only simple invocation interfaces without a contract.

In (A)CM, we have "arbitrary coupled open instances", implemented by humans, and the interface (goals) some kind of communicated. It can be augmented with templates.

But these two scenarios are not the only ones. The aspects a-d can be treated separately as required. Specifically, there are e.g. preplanned, goal-based coupled schemas possible, which are not at all complete and whose instances are abitrary coupable. And so forth. I.e. "structuredness" is neither a property of BPM or ACM solely.

@3. ...all options can co-exist but only one of them runs in particular moment...

That's not necessary! By decoupling interface from implementation the relationship between schema and instance (runtime, case) can be kept without loosing the ability to work freely with theses instances.

@4. ...Thus, if you say that the system is always ready for next cycle of processing, I do agree with you...

If the structure that couples the steps is a goal-structure (constellations), the difference can be clearly seen: controlflow based systems just insist of becoming certain preconditions true, and get stuck if an exceptional situations occur, (A)CM transfers control to humans, but the basic truth behind all this is always that one wants to reach certain goals, and devides the solution path into subgoals.

@6. ...resource/data centric... function/service centric...

Why centric at all? Aren't dynamic and structural features just two sides of one and the same enterprise coin? The reason, why we perform any kind of activity is we want to achieve a goal, and to achieve we goal we need to perform steps. These are inextricably joined.

7. ..."‘shared understanding’ of schemas"... decoupling because of inability or ignorance to preserve common/shared semantics

Yes, yes, yes...!

Interface: describes intent, and is coupled to changes in artefact state
Implementation: is exchangeable, but bound by contract, can be within monolith, can be a service

Andreas

Andreas, thank you for such informative response ( I see we are getting longer and longer, which not easy to read and response)

I have to say that I have not understood your comments:
@3. ...all options can co-exist but only one of them runs in particular moment...

That's not necessary! By decoupling interface from implementation the relationship between schema and instance (runtime, case) can be kept without loosing the ability to work freely with theses instances.

To me ( as the state machine works), an activity may be either in the process (structured) or out of the process - in the Case (unstructured) area. It is not about template vs. instance.


Also, I do not think that all process get stuck in the case of an exception. Some process are designed well and ALWAYS know what to do in the case of ANY exception.

The idea is to say a Step is not a compact unit (black box), but to consider it as a Class with an Interface.

Such a Step Interface is defined by
- it's name, e.g. "Application Review"
- a Purpose, which describes what the intent of the Step is, typically by specifying it's origin and it's target, e.g. "origin: an unreviewed application" and "target: a reviewed application", which is basically nothing else as the interconnecting artefacts along with there respective states
- possibly additional attributes (providing more context)

A nice side-effect of this approach is we can use derivation, too. So "Application Review Region North" might be modelled simply as derived from "Application Review".

If you want to chain such Steps, you do it by writing a sequence of alternating Steps and Artefacts[InState] (i.e. Constellations). This is the link to traditional CF-BPMS.

Next, classes can be instantiated. Here, we'd have "Application #4711 Review". This is our Case. So far, nothing else is specified but just the statement "application #4711 needs to be reviewed" - no implementation is determined! We only have defined a goal, which possibly is a subgoal in a hierarchy of goals in the enterprise. If in doubt, the "Master Step" always reads "Earn Money" (plus optionally "& Have Fun").

But it's left open how we do it. We can further subclass the Step and choose an automated system procedure Step, but we can also as well solve the Step manually, by adding further Sub-Steps (Activities), which are again solved by not-yet-specified means. Here comes SOA into play. This subclassing can be viewed as a kind of late-binding of final Step implementation, so to speak.

If I say "modelling", that does not necessarily mean a BPM wallpaper. The concept describes a metamodel, that can be used as a basis for a graphical modelling language as well as a basis for a software system and therefore links the painting world with the runtime systems and comprises also non-IT-scenarios.

@...not think that all process get stuck in the case of an exception...

Yes, of course not. I only wanted to express they CAN get stuck.

On behalf of Dave Duggal:

I knew you’d be tough enough to take on an objective discussion of ideas. : )
1. I was building on your comment re: ‘switching’ between unstructured action and structured process so I simply noted our system does that switching based on logic. That unified notion hadn’t been raised prior, but it sounds like we agree.
2. We had previously agreed Case has value, but itself is not process, nor are discrete uncoordinated unstructured actions. Think we are good on this one too.
3. I didn’t suggest you believed there was a ‘chinese wall’ or that you disagreed with Alexander.
4. We agree
5. It is ‘coordinated’ per your definition here, decision logic is consistent, but context data varies making outcomes vary.
6. Yes, we agree – related objectives, different architectures/methods, etc. It’s non-trivial as different implementations will exhibit different properties, which is the point I was trying to make.
7. I didn’t say semantics don’t matter. I was pointing out that using WSDL to pass schema documents adds some undesirable complexity to a SOA implementation. OASIS validated practice doesn’t equate to good practice, just a formally accepted norm, which is really a SOA workaround pattern as you are disregarding the fundamental role of the WSDL. From an objective architectural perspective this is a suboptimal arrangement even if it achieves a desired real world effect.
Re: Services. I agree IT isn’t really in control anymore. Consumers can consume what they want.

This leads to demand for custom services, afterall who wants to consume a generic service that’s less relevant? Because our system is Resource-based its artifacts are not encapsulated by interface, means they can be flexibly exploited based on execution context. This nets us what we call dynamically configured ‘Goldilocks Services’, which promote re-use (really re-purpose, which is more powerful).

8. Again, implementation matters because properties matter. One day I’ll show you the system so you can judge for yourself.

@Andreas

it is possible to create this or that object hierarchy for implementing Step however, from the architectural perspective, the logic is different, IMO.

I do not care how the Step is organised; I care WHY I need to do a step and WHAT Step is it. I have found two reasons for WHY: 1) predefined business logic dictates me that I have to (or do not have to) make a Step; 2) there is an event (including the result returned to me) I have to react to(I omit reasoning why I have to react). Now, the answer to WHAT the Step I have to take is not in what the Step does but in what result this Step returns to me. This is because only based on this result I can make a decision about the next Step, if any.

This logic is enough to me. Step is an Action result that I might or might not need next. Nonetheless, from the implementation perspective the 'world' may look differently.

Dave,

it seems we agree on all points. I only wanted to refer you to my next BLOG - Trying the Next Step When Doing an Adaptable Case Management - where I describe usage of logic/rules as the key element of the 'switch' - this is in response to your " I simply noted our system does that switching based on logic. That unified notion hadn’t been raised prior".

Thanks for putting that post through for me. I'll be happy if this goes through.

Re: logic driving interactions, just meant "hadn't been raised prior" in this thread. Using logic to advance a process is not new conceptually, I think what is interesting is the implementations and what they achieve. I'm glad you are pursuing the subject in your Blog posts.

Let me restate your comments in my terms:

> I do not care how the Step is organised

So you favor encapsulation. The Step interface suffices in that case, the implemantion stays hidden. Fine.

> I care WHY I need to do a step and WHAT Step is it

Even so fine. WHY is expressed by the originating and target Constellations of the Step, and WHAT is precisely captured by the Step's Interface identity.

> two reasons for WHY:
> 1) predefined business logic dictates me that I have to (or do not have to) make a Step

I.e., the Step is instantiated automaticallly by the system based on the fact that a certain condition was met, that might be because:
- a certain Constellation was detected
- a Token arrived (which is a special case of Constellation, too)
- an Event took place (which is a special case of Constellation, too)

> 2) there is an event (including the result returned to me) I have to react to (I omit reasoning why I have to react)

I.e. a person instantiates the Step (someelse or you yourself) because you're responsible, possibly with support of the system (suggestion who's responsible etc.)

> Now, the answer to WHAT the Step I have to take is not in what the Step does but in what result this Step returns to me.

I.e., you choose a certain procedure (implementation) for the Step, which still fits perfectly. You possibly instantiate further substeps (not final implementation) other people shall do for you or make use of automated services.

> This is because only based on this result I can make a decision about the next Step, if any.

Even here, the system and the preplanning can support you. By subclassing your Step, different possibilities and responsibilities can be distinguished. Even if you're the only one who can decide based on the result what to do next, there's no need to do something which breaks the architecture of the approach. Only who instantiates what differs, but that's the whole idea.

> This logic is enough to me.

Then you do not need more. In case you do (because there are results where the system can help more in making the decisions, because you want to automated more, because you want to understand and organise more etc. etc.), it is nice to know that the architecture you've choosen is open enough to support all possible scenarios and arbitrary, fine-grained mixtures of the approaches.

> Step is an Action result that I might or might not need next.

Yes of course.

So, I do not see any contradiction between your examples and the unified approach.

Andreas

Certainly, Adaptive Case Management is the same as the collaborative business process management. This concept can be used instead of BPEL4People - extension of BPEL for collaboration

I've found an instant & easy ACM (Adaptive Case management) & BPM (Business Process Management) software in one product at www.paydox.com.
Download & installation in minutes. WOW! Looks like blogs & collaborative business process management

Exceptional blog you guys have conserved there, I absolutely appraise your effort.

Leave a comment

In this blog, Michael Poulin writes about business and technology ideas, concepts, methodologies and solutions leading to service-oriented enterprise, the primary instrument for obtaining business objectives in fast-changing environments.

Michael Poulin

Michael Poulin is an enterprise-level solution architect working in the financial industry in the U.K. and the United States.

He specializes in building bridges between business needs and technology capabilities with emphasis on business and technical efficiency, scalability, robustness and manageability. He writes about service orientation, application security and use of modern technologies for solving business problems. He contributes to OASIS SOA standards as an independent member and is listed in the the international "Who's Who of Information Technology" for 2001. View more

Subscribe

 Subscribe in a reader

Recently Commented On

Categories

Monthly Archives

Blogs

ADVERTISEMENT