This is the third, and final part, of my blog posts regarding what we can learn from spreadsheets and non-traditional application development. The first part discussed how spreadsheets are a positive example of how line of business experts can develop their own applications. The second part discussed some of the challenges the BPM is in replicating the successes of the spreadsheet vendors, most notably the traditional enterprise software business model.

The essential point that I wanted to make with this series of posts is that, while BPM is unique in many ways, it does have some important parallels to other software categories. We, as the BPM community, should be doing a better job of learning from the successes and failures of the past. In this post I'd like to discuss several related technologies and the lessons that I think we, as BPM vendors, can learn from those technologies.

Spreadsheets

I'll start with spreadsheets, since that has been the example we've been discussing the most over the last few posts.

Things that spreadsheets did well that we should emulate:

• As I said in my first article, the model must be the implementation. Imagine if analysts had to send their spreadsheets over to IT where the spreadsheet is imported into a different tool, the cells "implemented", and an EXE generated. No one in their right mind would have used a spreadsheet if they had to go through all of that. I feel the same way about BPM tools that require a conversion step between an "analyst model" in BPMN and a "developer model" in BPEL. It's pure insanity.
• Quick iterations are important for business users. For non-technical users it is important to be able to get immediate feedback on changes. If a business analyst changes a process model they should be able to test the impact immediately. This immediate feedback both helps analysts learn the tool as well as prevents there from being a large integration and testing cycle.

Things that spreadsheets didn't do well, and we should just learn to accept:

• IT isn't going to be able to control everything. Most Excel spreadsheets are ugly hacks. But they work and they are "good enough" for the business users. Yes it would be nice if there was a re-usable library of spreadsheets with a sales forecast spreadsheet. It would be even better if every department used that same spreadsheet. But every department manager has their own way of doing things. (Which is why everyone is doing their sales forecasting in Excel instead of using the sales forecasting feature of the CRM package.) We need to treat processes as valuable assets but we also must accept that departments will make their own choices regarding their processes.

Business Rule Engines

What I consider a sister technology to BPM (and others consider a subset of BPM), Business Rule Engines have been moderately successful. While they haven't taken off outside of their key verticals (financial, manufacturing, heathcare & insurance mostly) they have generally been successful in delivering more control directly into the hands of line business. Like BPM Business Rules implementations also have multiple competing stakeholders and have to balance the needs of the business for flexibility against the scalability and manageability concerns of IT.

Things that Rule Engines do well that we should emulate:

• Rule Engines (in general) treat rules like enterprise assets: including tools for cataloging, managing, and searching existing rules.
• Rule Engines (in general) are good about managing the deployment lifecycle, coordinating the description of rules by the business, any additional development by IT, QA, and deployment.

Challenges that Rule Engines have that we should learn from:

• No matter how pretty an interface you have, line of business experts are not programmers. (More on this later in the week.)
• Standards (like JSR 94) aren't very interesting if the standards don't allow portability. (If you can't move your rules from one vendor to another the only benefit of the standard is to ISVs.) Rule Engine lock-in is very hard to avoid due to a lack of standards about rule portability.
• Standards (like JSR 94) aren't very interesting as a communication protocol if the standard doesn't include enough functionality to be comparable to proprietary alternatives.

Portals

A few years ago portals were the hottest technology on the market. Except that everyone had a different definition of what a portal was. Some vendors supplied complete out of the box systems while others vendors were essentially frameworks and toolkits from which portals could be built. Some vendors focused on security and integration, while others focused on knowledge management and publishing. Portals were really a conglomeration of multiple existing technologies where the integrated whole was greater than the many parts. Any of this sound familiar? It should, because all of these statements could equally apply to the BPM industry.

Things that the Portal vendors did well that we should emulate:

• Portals (in general) did a good job at selling the business value of their technology. Despite the fact that there were often multiple disparate stakeholders involved in portal decisions, portals have become a mainstay of enterprise technology.
• Portals, in general, did a good job at integrating with third party systems. Enterprise Portals, like BPM, typically require integration with third party systems in order to build meaningful systems. The leading portal vendors did a commendable job in making that integration possible.

Challenges that the Portal vendors have had that we should learn from:

• Standards aren't very interesting if the standards don't include enough functionality to be comparable to proprietary alternatives. JSR 168, while promising, has generally been a disappointment for real deployments due to its lack of support for intraportlet communication. And while more functional standards like WSRP are emerging they continue to lag significantly behind the proprietary APIs.
• Widely varying areas of strength. While this is really both a strength and a weakness, I believe that the reason that no dominant portal player has emerged is because there are so many different ideas about what is important in an enterprise portal.

There are lots of other examples out there, I'd be interested to hear some your own comparisons in the comments.

Posted by davidogren in BPM | Permalink | Comments (2) | TrackBacks (0)

So, in my last post, I was expressing optimism about how BPM tools could enable non-technical users to build their own applications. Just as the spreadsheet made departments capable of developing certain types of applications on their own, BPM would enable "the business" to develop their own process based applications. Today I'm going to to mix a little dose of reality and skepticism into the mix.

Let me start by referencing an old article by Tim Bray. Tim says that there are essentially two ways that technologies can get introduced into the market: the front door and the back door. "Front door" technologies are bought high up in the organization to solve a business problem or strategically reduce costs. Examples include ERP systems, CRM systems, and application servers. "Back door" technologies are snuck into the organization by sysadmins and programers to solve day problems or reduce tactical, day to day, costs. Examples include Linux, Perl, and and PCs. (Of course, technologies can cross from one category to the other: application servers used to be "front door" but open source J2EE servers have led to "back door" culture as well. PCs used to be "back door" but have become so commonplace that they are dominated by "front door" purchasing).

Spreadsheets were unquestionably a back door technology. A department manager expensed a copy of VisiCalc, 1-2-3, or Excel and wrote his own sales foreasting spreadsheet. Other managers found out about this spreadsheet and built or copied their own version. Management hates the spreadsheets because there is no way to "roll them up", because manager has their own one-off version, and because the data is vulnerable to loss and theft because it generally lives on the managers' laptops. So management tries to push a CRM based sales forecasting system. Success varies from organization to organization, but generally results in department manager's copying and pasteing from their spreadsheets into the CRM system on a weekly basis. :-)

One of the challenges of the BPM market, and the reason that I believe BPM has taken so long to bloom, is that it doesn't fit very cleanly into either category. On one hand, BPM delivers power directly to the business and is very "tactical" in delivering immediate solutions. On the other hand, BPM is sold in a traditional enterprise software model (high upfront license costs, enterprise salespeople) and requires the cooperation of IT because of the needed integration and maintainance. This requires a traditional "front door" approach full of RFPs and cross-functional teams conducting PoCs.

Hopefully you can see the conflict here. If BPM is going to be sold as enterprise software and integrated into IT systems it needs to have IT's blessing. A blessing which it isn't going to give if BPM will result in a rat's nest of processes that it has no control over. But if the business is going accept BPM (and give up the existing paper, email, or spreadsheet based processes) then BPM has to be a little bit subversive.

Now there are several ways to deal overcome this paradox. Some companies have tried the open source route. I believe this is fundamentally flawed as a strategy. The sysadmins aren't going to be able to sneak an open source BPM solution in the door the way they snuck Linux in the door. BPM requires too much enterprise integration and has too much end user visibility. Not to mention that open source has been traditionally weak at some of the key requirements of BPM such as documentation, end user appeal, and enterprise integration. (Please reserve your hate mail, I know that is a generalization and that there are exceptions. I am a generally a fan of open source.)

Other, generally low end, solutions have tried to appeal only the business, trying to fly under the radar of IT by staying departmentally focused and promising the business that they can operate BPM without IT. Others have attempted to give away their modeling tools, hoping to get the business bought in on the modeling tool before IT gets involved. I have to admit that I think this strategy is interesting. But it doesn't solve the fundamental problem, at least not by itself. To get to deployment, the business is going to need IT. 

Fundamentally, I think that the solution to making BPM successful is better cooperation between line of business and IT. I wish there was there was a technological "magic bullet", but I don't believe there is one. The business is going to have to realize that IT has value to add when it comes to writing business logic, organizing data, and working with enterprise systems. And IT is going to have to empower businesses to control their own departmental processes. Yes, that will mean that mistakes will be made. Departmental processes aren't going to look or act perfectly.

Which leads me to part three, which I'll hopefully post Wednesday night: what lessons we can learn from other technologies (including spreadsheets) regarding empowering business users. Including what to control and what to give up control over.

Posted by davidogren in BPM | Permalink | Comments (0) | TrackBacks (0)

I spent several hours stuck on the tarmac in Toronto today: the storms on the east coast kept us grounded. As such I didn't get a chance to finish the second part. Hopefully I'll post it tomorrow night.

Posted by davidogren in BPM | Permalink | Comments (0) | TrackBacks (0)

Several months ago, I started thinking about the parallels between spreadsheets and the BPM industry. Several of my customers had been asking me about which features of AquaLogic BPM that "non-technical business users" will be able to use and which features will require IT involvement. This is certainly a fair question because one of the core tenets of BPM is to give more independence to the business in defining their process applications. I had been hoping to finish that post this weekend. But, while researching, I found that another blogger had just made a post on the exact same topic. I guess there really are no original ideas. :-) :et me comment on the posts I found first and then follow that up with some of the post I've been working on.

The post I found was from Keith Swenson of Fujitsu responding to Tom Baeyens of jBPM. In short, Tom was arguing that analysis and implementation cannot be unified. That a "non-technical business user" may visually map the process, but that there will always be an implementation of the process activities under that process model. Tom says that "most often, there will have to be some custom code tied to the process either as node behaviour or as actions which are hidden from the graphical diagram." He also asserts that an a process model designed in an executable language (specifically BPEL) is less flexible for analysis purposes, asserting that BPEL has a level of specificity that the business analyst will not care about. I don't want to put words in Tom's mouth, but he was generally being skeptical of the mainstream BPM vendors and their claims of having business users develop their own applications

Keith responds by using the spreadsheet analogy as a counterexample. Specifically that when spreadsheets were invented they allowed business users to develop their own "applications" that previously required the intervention of IT. That even though Excel isn't a typical 3GL development language, it has been a critical tool for business in developing applications. (Keith specifically cites Sarbanes Oxley compliance and report generation as examples of applications which have been largely supplanted by Excel spreadsheets.) Keith says that BPM is similar. For a specific kind of application, specifically process centric applications, the spreadsheet enabled business users to be self-sufficient.

Obviously, since I had already been thinking of the metaphor, I agree with Keith'about the similarities between BPM and spreadsheets. And I generally frown on Tom's idea of creating artificial separations between analysis and implementation in BPM. The process model is the implementation.

I would take this metaphor one step further, however. Think about the way that enterprises use spreadsheets. There are many different ways spreadsheets are used. "Non technical users" have no problems using a spreadsheet. According to Joel Spolsky, a former program manager of Microsoft Excel, most Excel users were just keeping track of lists. One of the nice features of a spreadsheet is that simple things are simple.

But, of course, spreadsheets are actually quite complicated pieces of software. In addition to the simple "take the value from this cell and add it to this cell" basic usage there are all kinds of functions for gathering information from databases and web services, parsing text, validation, complex math, lookup tables, and conditional logic. Not to mention all of the features related to slicing and dicing the spreadsheet data including pivot tables, goal seeking, analysis, and reporting.

My experience is that a lot of departments have a few complicated spreadsheets that they have developed that solve specific problems related to that department: forecasting sales, calculating bonuses, reporting on field offices, etc. Most of the spreadsheets are pretty complicated. They were created by some "power user" that figured out how to use all of those conditional logic functions and hacked together a solution bit by bit over the years. I've also seen many enterprises leverage spreadsheets for core business processes, with spreadsheets developed by professional programmers using a combination of Excel's built in functions and Excel's macro language: Visual Basic for Applications.

So, which functions of Excel can a non-technical business user use? It depends on the user. The majority of users can't really use anything other than basic cell operations. But there are certain business users will learn the skills needed to access more functionality. But even for the "power user" some features (like database integration and Visual Basic) will require IT involvement. It's also notable that even though the average department can create their own spreadsheet, creating something that is easy to use, styled appropriately, and "customer ready" will often require user interface specialists.

It's the same with BPM. Some users are never going to care about anything other than the high level model and the key performance indicators. Others will learn how to create their own forms and business rules. And integration and complex rules will probably require IT. And it's also important to note that some BPM applications will require IT (or user interface specialists) just because of testing and QA. No one wants a compliance spreadsheet that works most of the the time or a order tracking BPM process that works most of the time.

More on this on Wednesday, when I'll discuss how well the BPM industry is addressing the needs of business users and Friday when I'll discuss what lessons (good and bad) we can learn from the spreadsheet industry.

Posted by davidogren in BPM | Permalink | Comments (3) | TrackBacks (0)

I was following some links this morning and discovered that Jesper Joergensen has started updating his blog on dev2dev. Jesper is part of our marketing team, but if you look at his recent post you'll see that BEA marketing folks tend to be quite technical. :-)

Posted by davidogren in BEA | Permalink | Comments (0) | TrackBacks (0)

I see that Sandy already beat me to posting about the (long overdue) 2006 Magic Quadrant for BPMS. I'm hoping that BEA gets republishing rights to this report so that I can give you a link. But in the meantime, I'll summarize in the same way that Sandy did.

As Sandy points out, we did enter the leader's quadrant with this report. In fact, BEA/Fuego leads the entire quadrant in "ability to execute". The other vendors in the leader's quadrant are Savvion, Pegasystems, and Lombardi.

Gartner basically missed a year in publishing the quadrant. So this report is extremely welcome. The BPM market changes quickly and the two year old data in the old quadrant was dangerously out of date. However, it's important to understand that the BPMS is still a poorly defined market. Each of the four leaders has a very different product with very different strengths and weaknesses. I hope to write more about that later this week, because I recently had a very harsh reminder of how different the focus of various BPM products are.

Magic quadrants are only high level summaries, of course, but we're celebrating at BEA this week. Being named a BPM quadrant leader and ESB Forrester wave leader in the same week is a good way to kick off the summer.

Posted by davidogren in BPM | Permalink | Comments (0) | TrackBacks (0)