In previous entries to this blog I outlined the need for simple, universal management practices that can be applied to IT as to other areas, in order to gain control over your IT infrastructure. Even if you outsource your IT development and support (in fact especially if you do), the only way to gain and not lose from new IT techniques and tools is to properly organize the humans responsible for implementing them.

How does this work in practice? I will look at an example, showing how introduction of SOA into an organization can be facilitated by focusing on management rather than on technology (the arguments are essentially the same for BPM, but I'll save a BPM example for next time). In this example, I'll compare the standard, technically-oriented approach to SOA recommended by leading vendors and analysts with a more business-focused approach based on management principles.

Let's start by assuming that senior management in a organization appreciate the potential benefits of emerging SOA techniques and tools, and are in principle willing to invest time and money in an implementation project. What is a sensible first step? Let's turn to the literature to see what is proposed. For example, IBM's publication Patterns: Service-Oriented Architecture and Web Services, part of their authoritative RedBook series, suggests (p.5): "When faced with the challenge of designing a solution for a business problem, the first step is to get a high-level view of the goals you are trying to achieve. A proposed business scenario should be described and each element should be matched to an appropriate IBM Pattern for e-business.", where "A Business pattern describes the relationship between the users, the business organizations or applications, and the data to be accessed."

The book goes on to give the following example: "Suppose an insurance company wants to reduce the amount of time and money spent on call centers that handle customer inquiries. By allowing customers to view their policy information and request changes online, the company will be able to cut back significantly on the resources spent handling this by phone. The objective is to allow policy holders to view their policy information stored in legacy databases."

Well, yes - perhaps. The trouble is, we are already deep into technical matters, and can no longer see the wood for the trees. At the very start of what is supposedly a high-level business process - using new SOA techniques to improve enterprise performance - we are already talking about "viewing things online" and "legacy databases". And as the methodology proceeds onwards, this just gets worse. The subsequent step is described as follows: "Once the Business pattern is identified, the next step is to define the high-level logical components that make up the solution and how these components interact. This is known as the Application pattern. A Business pattern will usually have multiple possible Application patterns. An Application pattern may have logical components that describe a presentation tier for interacting with users, an application tier, and a back-end application tier."

If I were a Financial Director, Sales Director or any other non-specialist-IT executive, my head would now be spinning so fast that I'd have given up any hope of understanding, let alone helping to control, what is going on in this project. Yet the involvement of experienced and skilled senior executives is vital in order to minimize the risk and improve the chance of getting useful results - and in a mission-critical project such as this one, with ramifications across all departments, the entire board should be able to discuss and advise on progress.

To be fair to IBM, their RedBook does state (p.80) that "Identifying the set of services that a business requires their IT to support is not a trivial task. Analysts agree that an approach or method is needed to uncover the business-aligned services, their dependencies, and their supporting large-grained components. Service identification is a determining factor in creating and migrating to a successful service-oriented architecture." In particular, a supporting article recommends that 3 separate approaches to "Service Identification" are used in conjunction to combine a top-down, business-driven approach with a bottom-up approach leveraging legacy investments: "Service-oriented modeling approach provides modeling, analysis, design techniques, and activities to define the foundations of a SOA [...] using a combination of a top-down, business-driven manner of service identification coupled with a stream of work that conducts service identification through leveraging legacy assets and systems [...] Finally, using goal-service modeling, you use a cross-sectional approach to cut down the sheer number of candidate services that might already be identified. A more judicious approach would be to first do top-down, then goal-service modeling, and finally bottom-up legacy analysis of existing assets."

However, we saw above that the top-down, business-driven approach tends to dive straight into detailed IT matters. And we can safely assume that a bottom-up approach leveraging legacy investments will be at least as technical. What about "goal-service modeling"? The article states: "The middle-out view consists of goal-service modeling to validate and unearth other services not captured by either top-down or bottom-up service identification approaches. It ties services to goals and sub-goals, key performance indicators, and metrics." Now we're getting somewhere - for the first time, there is mention of goals, and metrics with which to assess progress towards them. Business benefit seems to be entering the picture at last.

However, the proof of the pudding is in the eating. The example provided in the RedBook of such middle-out modeling is as follows:

Goal-service model for retailer business function:

1. Increase revenue

2. Increase sales

3. Provide self-service shopping capability

3.1. Purchase Goods

3.2. Source Goods

3.3. Provide user-friendly interaction experience

3.3.1. Get Catalog

3.3.2. Manage Shopping Cart

4. Maintain accounting records

4.1. Log Events

4.2. View Events

Help! How is any of this supposed to relate to high-level business objectives? It's either unhelpfully vague ("Increase revenue") or just as technical as before, if not more technical ("Get Catalog", "Log Events", etc). The lessons supposedly learned from this decomposition are described as follows: "Looking closer at the self-service shopping goal, it is important that interaction experience provided is user-friendly. To achieve this we need to provide the user with a shopping catalog that she can browse. In a real world scenario some sort of shopping cart facility would also be usual. This simple goal-service model helped us to identify that some additional services are needed to support the business goals." In other words, middle-out modeling helped us determine that we need catalog and shopping cart facilities: low-level implementation issues of concern primarily to software developers.

It is actually rather unfair to pick on IBM in this way. As discussed in previous blog entries, the advice given by other SOA experts is similarly technically-focused and disconnected from business matters. Despite the continual talk from vendors and analysts of how SOA (and BPM) are all about improving competitive performance, there is little help offered regarding how to ensure that such performance increases are realized.

A more helpful approach would be to start with business goals and leave out IT entirely. For example, here is a set of realistic strategic objectives, each accompanied by an example metric that assesses progress (in reality several metrics could be devised and applied for each objective, but let's keep it simple for now):

Note that this table is entirely devoid of technical detail - it describes elements of a business approach, not a set of IT systems. What is more, it offers an immediate means of assigning ownership to components of the problem. For example:

• The cost of sales objective properly belongs to the Sales Director, HR Director and CIO


• The shipping time objective properly belongs to the Operations Director and/or Logistics Director


• The 2 final objectives properly belong to the Sales and Marketing Directors.

Armed with this breakdown, it becomes clear that there are in fact a number of possible solutions. All these may utilize SOA development principles, but in a manner driven directly by business needs - and with measurable impact. To see this, let's consider the first objective - reducing human sales effort.

In order to maximise ease of use for the customer, Web-enabled self-service may only form part of a complete picture that also includes specialist sales representatives, to whom the customer can speak over IP on demand (for instance, by clicking a button on the Web site). It may also be possible to offer via the Web site packages resembling traditional supplier agreements (blanket purchase orders with constrained pricing and scaled discounts, for example), along with a conventional shopping cart facility. Further, it may be the case that a certain class of customer (in a specific geographical region, for example, or for goods of a certain value) prefer to place all orders over the telephone - in which case an option may be to outsource human sales effort, providing an alternative Web interface to salespeople working for the company - an interface with some similarities to, but also some differences from, that offered to customers. Other possibilities also exist, but that is enough to suggest how taking a high-level view of the problem opens up different options for solution - options that are more likely to differentiate the company from competitors than providing a catalog or shopping cart.

Supposing that a combination of such options are chosen for implementation, it is then necessary to ensure that they all work in tandem. The responsibility for this alignment can be handed to a particular senior executive, who will define in detail the strategies to be employed - and delegate the implementation of each strategy to an appropriate person. This is termed strategic control, and ensures that the high-level direction of the business remains the primary focus of senior executives. Details of process execution are delegated, like IT technicalities, to lower-level executives responsible for particular operational areas.

Each of these persons will take the strategic targets and measures they have been handed, and draw up a set of outline processes intended to achieve them. This is termed executive control, and ensures that the right type of process is used for each aspect of the implementation. What will these processes be concerned with? Their collective responsibility is to negotiate the necessary agreements with business partners, engage the necessary staff, put in place the necessary organizational structure, and make available the necessary system functionality.

Note that we are still a long way from IT detail. What we have arrived at is something much more business-focused - a set of outline processes, each of which involves a number of Roles, with well-defined responsibilities and channels for interaction - and which implement the strategic targets and measures handed down from above. Thus, when each outline process is handed on an appropriate manager (who will complete the definition and supervise the process itself) there are already mechanisms in place both to track the information required by senior executives for control purposes, and to feed this information upwards. And only at this subsequent level of business activity - termed management control - do people start to worry about the detail of what IT systems are necessary and how they will be implemented.

But what about integration issues? By splitting up the problem in this way, won't we be throwing the baby out with the bathwater, to end up with a whole load of new and incompatible systems?

In fact, quite the oppposite. What we are doing is to separate problem from solution - requirements from specification, if you like. It is quite likely that each manager with a process to implement will come up with their own set of desirable system features - which is exactly the sort of input required to an SOA project. SOA is all about finding commonality - the core set of standard services required to support different business processes. The only way to arrive at such a core set is to understand first what sort of business processes are required by the business. The executive with strategic control over a branch of our example project is responsible not for developing services, but for developing an organization and specifying services. In effect, the implementation of the services for reducing cost of sales, along with the implementation of services for the other high-level objectives described above, is a separate process to the business processes for the objectives themselves - a centralized, technically-focused process interacting with all the separate business-focused processes, to ensure that the services eventually implemented conform to the various sets of input requirements.

The key point is that the service development activity has been relocated where it belongs - at the end of the line, not at the start. Only once business strategy has been conceived, and refined through various management levels, do technical issues come into play.

TAKE AWAY

The promise of SOA is business agility. Not technical agility! The benefit of SOA as stated by all vendors and consultants is to deliver to the business an enhanced ability to control their operations, altering the way business is carried out as and when necessary. However, the approaches generally recommended for SOA implementation provide exactly the wrong basis on which to do this.

If IT systems built on SOA principles are genuinely going to respond in "real time" to changing business needs, their construction and maintenance must be controlled by business people, not techies. This means adopting simple, standard management techniques that can be applied at all levels of the organization - and hence that permit a seamless integration between top-level strategic objectives and low-level IT system construction. The way forward is to apply the techniques of Human Interaction Management.

In the next and final posting in this blog series, I will explore how these techniques can be applied to BPM.

Posted by keithhb in Business Process Management • Management • Service-Orientated Architecture | Permalink | Comments (3)

In previous postings to this blog I discussed how the next major IT challenge is to adopt new and powerful software tools without stacking up a set of business problems that could eventually lead to an unholy system meltdown. Many CIOs now see the need to integrate BPM and SOA with ERP packages and custom applications - and to do this across the board. However, few major organizations claim to have successfully got there yet. Current trends favour an incremental, softly softly approach - as Marc Fleury of JBoss says about SOA, "the reality of the field [is that] people are taking baby steps." And those organizations that are applying SOA across the board are already running up against serious maintenance problems - the experience of early adopters bears witness to major integration headaches not dissimilar in scale to those traditionally encountered in large IT shops.

BPM and SOA experts are, of course, aware of these issues, and in response urge that their technologies and techniques should only be applied with a strong business focus - in other words, the sponsors should not be geeks but suits. The idea is to specify solutions at such a high level that their implementation brings direct ROI to the business - there may be pain, but in this way you know that there will also be gain. However, while this is sound advice, little practical guidance is available on how to put it into practice.

In the BPM world, for instance, the technique recommended by most experts for implementation of process support can be summed up as follows:

1. Work with senior executives and selected middle managers to draw informal diagrams that represent at a high-level what they want to happen, taking little or no account of exceptional cases


2. Hand these informal diagrams over to the techies, to implement and deploy as "executable processes"


3. Repeat the above steps as necessary until the costs of the standard cases come down, using (for example) Six Sigma analysis techniques.

The consultants responsible for applying such a "method" are not generally worried by the 80/20 rule known to all business people - that it is the 20% of exceptions which give rise to 80% of the costs. Rather, they are confident that their particular proprietary bag of tricks (aka "best practices") is all-embracing enough to resolve any such minor quibbles.

A similar disconnect between business reality and IT implementation can be seen in SOA. The analyst firm ZapThink is at the forefront of SOA guidance, and their SOA Roadmap is a document representing the current state of the art. However, the roadmap starts with technical matters ("Wrap Legacy Systems In Services Interfaces") and continues with 9 more steps that are just as technically oriented. The aim and endpoint, of becoming a "Service-Oriented Enterprise", is arrived at without clear focus on the real-world business matters supposed to be driving the effort in the first place.

So if the business/IT disconnect problem is recognized so obviously as being important, why is it being handled so poorly by experts?

The reason is that the heart of the problem is something of a communal secret among IT people. In a large IT department not only team leadership but also the integration of strategic and tactical direction with lower-level management are highly problematic, as discussed in previous blog entries - but this is not made at all clear either to departmental managers or at board level. Executives whose experience and focus lie in business (rather than technology) matters tend to associate problems of IT infrastructure maintenance with unavoidable technical issues, rather than with issues of multi-level people management, since this is what they are told by their IT staff right up to CIO level. Those who have worked in a typical IT shop, and experienced the human issues first-hand, tend to cover them up for want of a solution, preferring to put the blame on technology issues that are supposedly beyond their control - and when the human issues become too manifest to keep a lid on, the general approach taken is to outsource the whole lot, and gain short-term cost benefit in exchange for a longer-term reduction in flexibility and control.

Outsourcing has its benefits of course, but it is not a panacea. In particular, you cannot remove the strategy disconnect in such a way - rather, it will be exacerbated, since there are now contractual barriers between the decision to do business differently and the implementation of corresponding IT systems.

So how will the formal approach to people management recommended in the previous posting to this blog sort the problem out?

The answer is that, like SOA and process choreography themselves, it is based on:

• The agreement between multiple parties on behavioural contracts


• The assumption that these contracts may be renegotiated on an ongoing basis.

In particular, an executive with "strategic control" over a business domain provides to their subordinates a set of targets, along with metrics that measure progress towards these targets. Each manager charged with such targets and metrics is then responsible for instigating appropriate processes via "executive control" (they have a contract with their strategic controller to do so, based on targets and measures). The team or group leaders with direct "management control" over specific processes must then facilitate the operation of these processes from within (they have a contract with their executive controller to do so, based on outline process descriptions).

The different components of this pattern may well be repeated - for instance, some individuals may find themselves responsible for more than one of these levels of control - and any of the contracts at any level are subject to change at any time. However, the simplicity and formality of this approach means that it can be implemented at all levels using the same simple diagramming techniques and people management practices, and the ongoing changes that are inevitable in modern business practice can naturally trickle down through the layers, from board members to programmers. In future postings I will explain how this works via examples of BPM and SOA implementation.

TAKE AWAY

Though it may seem like common sense to view business life as a set of changing agreements between colleagues, this approach is rarely applied in practice. As a result, human issues bedevil the management of large-scale IT operations - and worse, such issues are generally concealed from the sight of non-IT people by those who have to live with them.

This is unnecessary. There are simple, universal management practices that can be applied to IT as to other areas, and the result of doing so is to gain control over your IT infrastructure. Even if you outsource your IT development and support (in fact especially if you do), the only way to gain and not lose from new IT techniques and tools is to properly organize the humans responsible for implementing them.

In future postings to this blog, I will explore how to put this into practice.

Posted by keithhb in Business Process Management • Management • Service-Orientated Architecture | Permalink | Comments (0)

In previous entries to this blog I have discussed how a new breed of powerful software development tools is giving rise to another generation of inconsistent departmental applications and explained why the root of the problem lies in the informal application of diverse "best practices" to management. In other words, for a large organization in particular, the only way to regain control of your IT infrastructure is to apply a small number of general-purpose management principles consistently. First find a clearly-defined and self-contained set of management principles applicable to any kind of work, whether carried out by senior executives or junior programmers - then apply these across your organization at all levels.

Only in this way can you guarantee that the strategies and policies agreed at top level will be implemented at lower levels - and only in this way have you any chance of finding out what the status of such implementations are. Do your organizational leaders apply techniques such as Balanced Scorecard, EFQM Business Excellence, Activity Based Costing, Economic Value Added, and so on? Are these techniques seamlessly translated into work practices and resulting metrics at the lowest level?

In most cases, the answers are yes and no. Whatever senior executives may be told by their subordinates, by the time their guidance has been handed down through several levels of management and matched against existing operational systems, the chances of it getting carried out as intended are fairly low. The result of a new initiative is often simply to create more work, as people struggle to extract new statistics from old applications. The problem is not the guidance itself, but the fact that there is no management structure in place that is general-purpose enough to implement any such guidance.

The problem is not specific to IT, of course. Encouraged to "move with the times" by exposure to continually changing advice from keynotes, consultants, business bestsellers and the press, the fear of being left behind and desire to encourage initiative make it hard for a large organization to maintain management consistency in any area of operations. Yet in IT the stakes are particularly high, since it is getting ever easier to create complex and expensive disasters.

Of course, in a modern business environment, a particular set of management principles will only be a candidate for such across-the-board adoption if it is founded on the capacity to allow natural change to working practices on an ongoing basis. For instance, mechanisms for encouraging and implementing feedback are required to support the development of a learning, creative organization, and this makes (for instance) old-fashioned command-and-control styles of management unsuitable for 21st century business leadership. Work itself can and should change - but its management should not.

The heart of the matter is to distinguish the various forms of control that can be applied to work, of which there are 3.

1. Strategic control is the definition of how work is to be measured. This is the what senior executives do when they apply (for example) Balanced Scorecard. However, strategic control may also be applied at lower levels. The fundamental output of strategic control is a set of metrics that indicate to other people the basis on which their activities will be judged - though it is up to the people concerned to decide how best to meet these criteria.


2. Executive control is the sponsorship of specific work processes - initiating a project, organizing a bid, commissioning an evaluation, ordering a relocation, and so on. The essence of strategic control is to define the Roles applicable to a new work process, specify their responsibilities, and possibly to mandate that there are certain Interactions between them. Executive control can be applied at any level of an organization.


3. Management control is the ongoing facilitation, monitoring and adjustment of a work process - from within, not from above. Though a person responsible for management control of a process may have line or project management authority, the essence of their Role is to work with all those engaged in the process. Again, management control can be applied at any level of an organization.

TAKE AWAY

Does your organization have a seamless means of translating the strategy visions of senior executives into the working practices at grass roots - a means that ensures the natural co-evolution of work processes and operational systems?

Or do you see people struggling to comply via inconsistent management practices and operational systems, with consequent waste of time and resources - not to mention increase in stress levels, and increasing disconnect between working patterns and application system interfaces?

If the second of these cases has an uncomfortable resonance for you, stay tuned to this blog.

Posted by keithhb in Management | Permalink | Comments (0)

In the previous posting to this blog, I discussed how the new breed of powerful software development tools and techniques are, in fact, leading many large organizations down a very dangerous road.

In my consultancy work I have often seen organizations in which there are many different software projects covering related territory, yet without either the technical or managerial integration necessary to ensure consistency. In extreme cases, the organization itself is effectively partitioned by the very software developments which should be uniting it. Not only is this wasteful and inefficient, but in an age where statutory financial controls are being applied ever more fiercely, the existence (for example) of multiple conflicting systems of record may even lead to the downfall of the organization in question.

In the previous posting, I suggested that there is a parallel between current problems with the organization of software development and the problems of 2 decades ago with the practice of programming itself. Back then, the focus of computer science was on applying mathematical and logical techniques to improving how people wrote programs. Now, if we are to start reducing the problems caused by the explosion of new and powerful software development tools, we need to apply such techniques to improving how programmers are managed.

Is this starting to sound tediously technical to you? You may be thinking that maths and logic are the sort of things academics like to write papers on, with little relevance to the real world. Not so, unfortunately. If you have any experience of supervising, or just working in, organizations in which more than a small number of people are developing software, you will have seen some of the problems - and need to understand how to avoid them.

The crux of the problem is that large-scale collaborative efforts are unlike small-scale ones. The mathematical study of chaos theory shows how a large number of simple activities can, when taken as a whole, give rise to very complex behaviours. In order to better manage software development work, it is necessary to understand 2 key aspects of this.

First, there are tipping points. The French mathematician Rene Thom developed Catastrophe Theory in the 1960s to study how a complex system is subject to sudden changes in overall behaviour. This is a very everyday experience - one moment everything can be going fine, then suddenly it seems to be in crisis mode with no way out. Or vice-versa. What is going on? And how can it be controlled?

Thom's notion of "fixed point attractors" describes how a system may have specific points of equilibrium - points from which it can be hard to shift the system, yet between which the system may jump very suddenly under certain conditions. In a simple case, a ship can be either upright or capsized. Another classic example is equity trading, in which the value of a stock may shift dramatically in a small period of time. Further, a system may move away from an attractor without arriving at any fixed position at all - financial markets can easily descend into complete chaos, for example, which happens every now and then as we all know.

Mathematically, one can show that there is only one way to guarantee that such sudden changes will be avoided. This is to reduce the number of "controls" over the system - the number of factors affecting its behaviour. If there are too many controls, it is effectively impossible to ensure that the system does not suddenly flip from one point of equilibrium into another, and a descent into chaos at some point may well take place. However, most organizations rely for management guidance on a mixed bag of "best practices" as supplied by their various consultants, together with whatever tips people may have picked up from the latest business bestsellers - exactly the surfeit of varying control types that is most likely to lead to catastrophic behaviour. What is required is a systematic, structured approach to management, applicable at all levels and to all types of work.

Turning to the second lesson from mathematics, Benoit Mandelbrot and his "fractals" are famous for the beauty of their graphical representations. The notion is that one finds certain patterns in nature - coastlines, tree branches, and nerve fibres are among the many examples - which are built up by the repetition of a similar structure at each level. A map of a coastline at any scale will reveal the same sorts of indents and projections: bays and peninsulas have a similar shape whether they are tens, hundreds or thousands of square metres in size.

Importantly, for our purposes, fractal patterns can be observed in social life too: the same relationships and qualities apparent in small teams can be applied to large organizations and even nations. What is more, the theory of fractals shows that the application of such patterns at a small scale leads naturally to the emergence of the same patterns at a high level. In other words, if you want an organization motivated by certain aims and guided by certain behavioural principles, start small - implementing such ideas at grass roots level is not only more likely to take effect than mandating them from above, but may in itself ensure that the aims and principles are applied also at higher management levels.

TAKE AWAY

Taken together, these lessons from mathematics are the key to gaining control over the large-scale collaborative work typical of organizational software development. In future blog entries, I will explain in more detail how they can be implemented. For now, ask yourself if software development work in your own organization is guided in a structured way by a small number of sound management principles, applied equally at all levels. If not, there is trouble in store. It may even be here already, without you knowing it - if your organization is one of the many that lack either a consistent means of detecting management problems, or the means to deal with them once they have been detected.

The good news is: it is possible to rectify even the most apparently chaotic situations. Guided by the application of mathematical principles to business theory (and taking in a range of other disciplines along the way), control over any collaborative work can be regained simply by applying a small number of management techniques in a consistent fashion. To find out more, stay tuned to this blog.

Posted by keithhb in Management | Permalink | Comments (0)