We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.
Start a Discussion

How big a problem are 'dark processes'?

Vote 1 Vote
As Jim Sinur writes in this blog, "A dark process is an unofficial process used to deliver results and not visible to management." So how big a problem are these processes, and what should be done about them?

15 Replies

| Add a Reply
  • My first reaction was 'WTF is a Dark Process ?'
    My second reaction was 'if they have the same effect in the enterprise as Dark Matter has on the Universe then what is the fuss about ?!'

    Instead of reducing these processes we should expose and understand why they exist in the first place. If they exist it's because our so called 'best practice' is anything but and removing them erodes business success. Perhaps Dark Processes are the next best practice we can find, and rebels are responsible for their existence.

    Poor Jim fears them both.
    I say embrace them both.

  • Theo has it exactly right. We have to expose the dark processes. Just because we can't see the dark side of the moon, doesn't mean we don't know it's there. With a little bit of effort we can also see what's on it!

    Same here for Dark Processes. This is where Desktop Analytics takes a lead. It uncovers everything that is executed by a user, by every user. Since it's looking at Desktop Activity, 24x7, there are no longer "dark sides" per se'. Up until very recently, this type of Desktop activity monitoring has only been achievable by sampling (stop watch / lean 6 time and motion etc.,). Think of it has worker analytics. It is a game changer and essential in successful BPM. http://www.openspan.com/products/desktop_analytics/

  • Dark processes....isn't that another attempt of vendors to sell their process mining stuff?

    I thin dark processes are the coolest things on earth. They are the processes that are not designed in the ivory process tower by visio therapists.

    They are the processes that really help customers executed by empowered, caring, employees (with guts)

    Indeed, dark processes must not be deleted, they must be loved and cherished!

    And now all sing:

    We love you DP we do, we love you DP we do!

    Always look on the dark side of processes, toedoe, toedoe, toedoedoedoe.

  • They are a problem, and an opportunity, and probably bigger than we think. Most of what we do in a social setting (including work settings) is not explicitly known. Even the people who perform as part of a dark process don't necessarily know they are doing so. Like riding a bike, the exact movements necessary to keep your balance are not consciously known.

    Jim's discussion of dark process reflect more the increasing power of process technology to support processes, and the desire to reach beyond the old fictions of what an organization does, and attempt to fill in all the details. Like the Heisenberg uncertainty principle, we will reach a point where the day to day fluctuations necessary defy codification. There will always be people who insist that there is "one right way" that everyone should perform work, and uniformity is itself an end.

    However, creative people and innovative organizations will be empowered to solve ever changing problems around them ... and those believing in the fiction of uniformity, will call that work "dark processes".

  • Dark processes are real and can cause great pain to us all. The mortgage selling process where important information is either not collected or misreported with no checks is a good example. This went all the way through the subsequent trading of packaged financial instrument where the real risks were just not known. And so the financial meltdown in born; it does not get much darker than this for us all.
    However these "dark processes" are very common where the centralised driven IT systems are so clunky and unfriendly for users that off line activities take place to create information with little real control. Not only that but black processes tend to allow knowledge to remain in people’s control making companies vulnerable in a variety of ways.
    It is important that people at work are empowered to use initiative and most do this for the good of all but the minority will see opportunity to exploit where sensible controls are deficient. Another dimension is compliance which is just going to get stronger at the point of information creation. This needs a whole new approach from software to support both these seemingly conflicting requirements. It clearly needs the BPM mindset and the fast emerging ACM see interesting thoughts from Max Pucher http://isismjpucher.wordpress.com/2012/01/04/acm-and-bpm-a-battle-of-hemispheres/ The old software models are just not up to this important job as Max says “ACM creates a real-world perspective and transparency on what is actually happening related to our goals. This is why I see an ACMS as the software engine of an embedded, goal-oriented BPM methodology” And so we can harness those “black processes” ......turn them to gold!

  • Let’s look at Jim’s definition “A dark process is an unofficial process used to deliver results and not visible to management”.

    So “unofficial process” means that these are not identified, recognized, sanctioned or supported. We know they exist but they are not “visible to management”.

    It doesn’t make them right or wrong. Yet.

    The key is the visibility to management. That would bring them into the light. Email is probably the best evidence of the existence of these dark processes. How many “processes” in your organization are done in emails? How many conversations are trapped in your inbox with information that would help improve business productivity if it was “visible to management”?

    That will tell you how much of a problem the dark processes are in your business.

    The reason why people revert to email is because of the rigid nature of flow-based process solution where you have to define the exact path of process upfront. It works for a number of rigid processes in an organization but not for those unpredictable processes where people (and most often customers) decide on the next step in the process. I suggest that you read Keith’s “Mastering the Unpredictable” if you want to understand the increased interest in event-based process solutions and adaptive case management solutions.

    The shear volume of dark process information necessitates the use of technology tools. It is not tool vendors looking for a new market. The problem (and market) exist, we are looking for a way to shine some light on these dark processes. It can be through an event based BPMS or through case tools. It is also through process mining tools linked to an event based BPMS, for example, that you can look at real historical process paths that these unpredictable processes take. It gives management visibility at both an individual transactional level as well as at a aggregated level to identify process flows that may impact operational risk, for example.

    I use an example of a Warranty Claim process for a high value item in a blog post that I recently did. It highlights the fact that the multiple outcomes that exist are not necessarily wrong. Each process path and outcome suits the unique circumstances and context of each case.

    More and more businesses recognize this and some are starting to see the light in the dark tunnel where dark processes live.

    Anyway, enough said, let me get back to my email inbox…

  • I tend to agree with Peter point of view rather than others opinion expressed.
    Dark processes discussion is not about a crusade against the creative side of execution and put people obeying to corporate dictatorship. I found somehow tired this continuous argument that companies to succeeded they need to let people define its own path when they sense work must be carried. Just imagine what is let people decide how to pay invoices and the variability that it would cause.
    Hence, instead of collecting those kind of arguments that Steve Jobs and others enabled creativeness to a maximum extreme to everybody benefit (or some lunatic failures like the Next computer I used before), dark processes is about bringing visibility of what is actually happening inside and outside enterprise borders.
    Jim's original post is a new version of original by Wil Van der Alst called Desire lines or cow paths, about the importance understanding if current enterprise practices are better than deviations or contrary the pirates found a better way to do daily job independently if is scientific or a basic and routine work. I prefer knowing the truth than live in the illusion. The shifts in working practices and new technology is making a profound shift how work is being handled. Thus, is becoming much more difficult to understand reality. Are you willing to live in darkness? Or you want to embrace new analysts approaches like Process Mining that shows you the reality independent of the process type played (structured, adaptive, dynamic... ) and agnostic from technology used to work. http://ultrabpm.wordpress.com/2011/12/19/a-process-mining-project/

    • Alberto,

      Every process is unique and yes, things like paying invoices (who actually started to call this accounts payable?); please standardize those things that will give you no competitive advantage. Completely automate it. Ready.

      But I believe in primary processes creativity and empowerment is more important. And I don't understand why that should be hidden or you should be afraid of it?

      Who says that another way of helping your customer (I am tired of the term dark process) is hidden? In my company we are very open about it. We chat and blog about it, we discuss what worked what didn't work; nothing dark about it.

      But for sure, it also might depend on the type of business you're in.

      So I don't believe it is always dark just a little foggy. Education on real BPM, Better communication, no dictator CEO and no compliance driven work might be a start to clear that fog.

      And your argument about process mining: That only works when the dark processes are somehow executed in a system that produces log files. Do you think that is true for all dark processes? And if mining tells you that the system offers so much freedom, why was that freedom put into it at first?

      But, I think most of the 'other ways of helping customers' can't be 'discovered' with process mining because there is no such log information.

      Oh yes, process mining is a great technological innovation with all those moving balls and nice analytics.

      Some projects (with useful log information), look like a success, but to me it's still too much an academic toy....in a nice marketing suit.

      So, dark processes: finally they really seem to be invented to sell process mining stuff ;-)

      • Emiel:
        Process Mining is a technology that can bring enterprise visibility out of the catacombs and is more than an academic project as more enterprise examples are seeing the light of the day. Also is not system log dependent as it can be played using for example natural language processing. Nevertheless, what is your prescription? Go down to the catacombs with torch and try manually to understand reality?

        • Hi Alberto,

          Like your metaphor of the torch in the catacombs. But as I have a different opinion on starting with managing your organization by process I wouldn't even bother going into the catacombs (actually I am afraid of spiders;-)

          As seen in earlier discussion I like to use soccer metaphor for process management. The only place where it must happen is on the playing field. So if process mining techniques are used to show you what is happening NOW on the field (and you are able to adapt) it is great. But then it has become just a BI-Monitoring tool.

          The argument against this 'processes need to happen live-opinion' is that after the game you can use process mining techniques to analyze your game and see what can be done better. And that is true for soccer because the game is (for example) videotaped. And the game it is the thing you want to videotape.

          But when you do this for dark process you never know if you videotape a process and the right information about it to really tell something useful about process performance.

          And if that is the case in process mining (it gives you useful information how you managed your real process), you don’t need mining anymore but just a BPMS with proper Process analytics.

          Then mining has become a ‘normal’ BI tool.

          But what mining claims now, is what most people like to call ‘discovering processes’ out of nothing. But that ‘nothing’ might not tell you anything about process performance.

          Back to the metaphor. Just Mine a soccer game out of nothing might end up with information like:

          - The ball bounced 1.205.268 times
          - The referee had a blue shirt
          - Average body temperature of players was 38 (Celsius)
          - Total goals scored were 5
          - So each team scored 2.5 goals on average

          So, all the information collected when using mining to do so called ‘process discovery’ might be completely out of process-focus because the Data you are digging in might never be set up for process management.

          So that is why I truly believe organizations first must understand what it means to manage by process and then apply the tools that might help them to bring this to a higher level. And then I see the most value in mining in ‘live process analytics’ and later on (when real processes are played) as improvement tool.

          So I would like it when all the (great) techniques are used to really manage by process, not as an improvement tool. I know several parties are working on that.

          Back to your torch.

          Let's say that process mining is a 25000 watt led light in the catacombs. Wow, I will really see a lot. But I am absolutely not sure that I understand it, that it is useful and that it tells me anything about process performance.

          And just another thought. Won't dark processes operate like rats? When you put a bright light on them they just disappear ;-)

          Happy processing!

  • very interesting dark process analysis, I am just connecting the dots here: Dark Process, Shadow IT or Architecture Debt.....though it doesn't mean dark color is always bad, as matter of fact, it means the business takes initiative to adapt to the changes promptly via bypassing bureaucracy;

    However, as we have many quality discussions on how to manage shadow IT via enforced GRC discipline, I think similar effort need be put into brightening dark processes, as some maybe just the best practices for process improvement or innovation, it can be amplified and shared at enterprise scope, but some of them cause business problems, need be removed.

    The following question would be: If ignorance is not a best choice, should CoE BPM takes responsibility of them, or EAs work on them, or should the new committee including all related "sunshine" parties to transform them accordingly. thanks.

  • The comparison with Dark Matter is a very good one. I wrote a post on the Higgs a little while ago: http://isismjpucher.wordpress.com/2012/07/18/process-maturity-versus-the-real-world-of-the-higgs-boson/

    HEY, THIS IS FUN: Dark Matter is a hypothesis of cosmology to keep our current mathematical models sane. There seems to be a kind of gravitional effect in existence that under current hypothesis means that there must be matter that 'causes' it. As we don't see the matter we call it 'dark'. There is also dark energy, BTW. I propose actually that there is no matter that 'causes gravity' because something has to be there first to cause something else. As it is the Higgs effect that 'causes' particles to acquire mass, does the Higgs cause gravity? Or is gravity just there and there is no mass (=matter)? Particles scoop up energy through the Higgs effect from the vacuum (zero-point energy) and kind reduce its tensor energy that we might (my proposition) perceive as gravitaional 'bending' of space. Which means that energy is not equal energy but they have opposite signs! Means that certain kinds of energy cancel themselves out. If we change our mathematical model there is no longer the need for any dark mysterious stuff. I am just challenging conventional thinking, not saying I know better!

    How do we apply that to dark processes? Exactly the same way. It is not the 'dark process' that is wrong, it could be the model! The illusion that there are processes that run a company requires us to see the stuff we don't understand as 'dark'. Maybe it is the 'BPM experts' that are wrong about the model?

    My proposition: What is called dark processes is simply what keeps a company running against its organizational restraints. Companies are run by people who cooperate willfully towards an agreed upon purpose! PEOPLE COLLABORATE! That is process. PEOPLE who follow flows don't collaborate and they don't care about achieving goals. BPM experts propose that when you don't have a flow then there will be chaos and bills won't be paid. Most bills aren't paid today because the money isn't there not because there is no process. The money isn't there because people don't collaborate towards the right goals. Money is like the Higgs field and particles (units-of-work) travel through it to acquire cost. Control is higher cost than the unit-of-work. To keep our false process-driven budget models sane we now have to claim that there are dark processes that waste the money. Hmm ... sounds familiar?

    What executives under current business process planning models try to do is to turn a business into a Large Hadron Collider (cost $9b to build) to smash few protons into each other and measure it to proof that the model is right. According to systems theory it is the control that wastes the money. If each particle knows by itself where to go that is the most efficient way (called FERMAT's PRINCIPLE). So we need transparency, but not 'bottom-up' by discovering dark processes and trying to control them, but TOP-DWON TRANSPARNECY by empowering people with authority, goals and means.

  • Big enough to:
    1. Create a hot seat for the CEO,
    2. Facilitate risk to internal and external accountability,
    3. Thwart the ability of integrating silos, IT and knowledge workers in supporting business decisions,
    4. Encumber efficiency in minimizing and managing risk,
    5. Create risk to market share
    6. Etc.
    If you can’t see it, you can’t manage it or fix it, more than likely you don’t you have it. Processes are created to facilitate a need to get work done, whether strategically or adhoc. A health check of how work is done, why it is done and the resources available to facilitate getting it done; the entire value stream needs assessing to include the technology, the needs of knowledge workers - an inside/outside review of the enterprise.

Add a Reply

Recently Commented On

Monthly Archives