My personal reflections after the conference focused on open source.  Regular readers of this blog will be expecting the third instalment in the current series on the real-world business issues of SOA.  However, it's the week before Christmas, so I will defer the heavy stuff until the New Year, and instead share with you some thoughts on what is happening at the leading edge of software production.  And these thoughts are appropriate for the time of year, Christmas being a traditional time to take stock of how some things change and other things stay the same.

I am now aged 43, and have been working in the software industry for nearly 20 years.  Many people who enter the industry as developers have moved on by this stage into less technically-oriented roles - consultancy, management, or writing, for example.  In fact, I have played such roles myself for a long time now, but make sure also to periodically return to the coal face to refresh my development skills and actually build software, as I am doing at present.  However, this is actually quite unusual - by their mid-forties, most people who started out by programming are no longer developing software in their daily work.

The result of this is that developers have, as a community, quite a short memory - and so tend to be unaware of long-term cycles in best practice.  For instance, few developers now remember why the 1980s saw a general move away from procedural programming towards object-oriented programming.  Hence, they are unaware of the deficiencies in the procedural programming techniques underpinning SOA and BPM, as evinced by languages such as BPEL.  Similarly, few developers now seem aware of potential issues with weakly-typed programming languages such as Javascript, or consider the implications of using an old-fashioned mainframe client-server programming model such as AJAX.

Further, a corollary of the youth of programmers is that they tend to take little account of such warnings, when issued by greybeards!  This is healthy in some ways, but it is also worth remembering Santayana's famous words on how those who don't remember history are condemned to repeat it.

An interesting example of such cycles is the current stir around open source software.  During the Javapolis conference I met and talked with various people engaged in producing and/or using open source software.  And it was fascinating to hear not only their evangelical enthusiasm for open source, but also their concerns.  These concerns centred around issues such as the following:

• Growing and retaining a developer team
• Growing and retaining a user base
• Maintaining code consistency and quality
• Preventing feature cherry-picking by competitors
• Monetizing products
• Retaining control over products

Sound familiar?  There is nothing at all in the above list to differentiate open from closed source.  All software vendors have these issues.  And therefore, by association, so do their customers.

Many people seem to view open sourcing software as a solution in itself - both a solution for vendors (to gain a community) and a solution for customers (to lower costs).  I don't see this happening.  In every area of life, you get what you pay for, and enterprise software is no exception.  Complex issues of software development and use don't magically go away - they just pop up in slightly different forms.

In fact, the discussions I had at Javapolis made me wonder even what it means to say software is "open source".  Is it just about access to program code?  Hardly.  Enterprise customers of commercial software vendors have always been able to get hold of program source code if/when they need it, either by licensing it or by arranging for it to be held in escrow against the supplier going out of business.  Further, most enterprise customers view access to the source of third party applications as a last-ditch option they hope and pray never to need.

So is it about price?  Well, again no.  Free software may well be closed source.  Further, enterprises always have to pay for any software they use, if only in service costs - which have always been been far greater than license costs, as Eric Raymond pointed out so clearly in his seminal work on open source, "The Cathedral and the Bazaar".

How about licensing?  A tricky one.  There are many variants of open source license, and much debate.  However, it is fair to say that most producers of commercially significant open source code are very careful about the license they use - and about the way in which their software is released into the marketplace.  Of the concerns listed above, the number one (it seemed to me) was retaining control.  Few people really want to let go of their baby, especially if you have invested serious time into it, and that baby is now worth serious money.

Finally, is it about a difference in the way software is written and/or released?  Here again, I see no real difference any more.  Most significant open source projects and commercial software packages are gravitating towards a halfway house between the agile coder's "release early, release often" mantra and the "seasonal release, periodic patch" approach characteristic of traditional closed source.  It is in neither the supplier's nor the customer's interests to adopt either extreme.  OK, you can download a nightly drop of Eclipse, but only the diehards will install and try using this for their daily work - and very few people indeed would do so with an unstable release of JBoss application server.

In fact, I would argue that what "open source" is (or was) truly about is the community model of development, in which people from outside the boundary of a single organization actively contribute to the application, and engage with the developers to test it.  This model is what I now see breaking down.  Most successful open source applications these days are entirely controlled by a single commercial vendor - Sun (Java), IBM (Eclipse), RedHat (JBoss), and so on.  Nearly all, if not all, "committers" to such open source projects work for the company concerned.  So how are such applications genuinely different from Windows or WebSphere?

I think we may be over halfway through a long-term cycle.  The pendulum is already swinging back, away from open source and back towards more old-fashioned models of software production. It is quite possible that as we go forward, it will mean less and less to label a software application as "open source".

Just some seasonal reflections - no TAKE AWAY section this week!  Your comments are welcome as always.  And have a very Merry Christmas.

Posted by keithhb in Open Source | Permalink | Comments (2)

In last week's post, Is your SOAP turning to SOUP?, I described some of the problems in store for SOA adopters.  In particular, I asked how your organization is dealing with service change, and how it is measuring the ROI into specific services.

Surely, you may ask, these issues are old hat?  We've been dealing with such questions forever.

The point I am trying to make is that the ballgame is changing.  Once upon a time, IT departments met the needs of the business by building custom applications and/or implementing packages.  Periodically, they had to justify enhancements/upgrades to these applications/packages.  And every now and then, they would decide whether or not the service provided by such software was still delivering value to the business, or whether it should be abandoned/replaced.

SOA makes this process much harder.  Once your monolithic software applications have vanished beneath a service layer, you can no longer conduct such analyses on each application in isolation.  Rather, you have to look at a much bigger picture - one in which the inter-relationship between software components and business usage is vastly more complex.

And even this is not the whole problem.  There are yet deeper issues, related not simply to justifying software changes but also to implementing them.  Sometimes there are dangers that are obvious neither to developers nor to the business.

Let's take an example.  Bear with me, since detail is necessary to illustrate a typical problem.

Suppose you work for a financial services company, and your leasing application has a useful piece of code for calculating credit risk.  The calculation is long and involved - best of breed, the result of years of refinement.  Further, it makes use of an expensive third party subscription service for which you pay a flat annual fee (as many checks as you like during each year).  Your developers observe that this code could and should be made available as a service for use by other applications, not only in the interest of maximizing benefit and cost-effectiveness but also to make such calculations consistent across the organization.

So the module is refactored to expose the credit risk calculation as an independent function, and this function is then exposed as a service.  After checking that in each case that the calculation is appropriate for various other applications, they are each refactored to call this service, and their own credit risk calculations scrapped.

So far, all well and good.  However, no-one has noticed that an interesting aspect of the leasing application's credit risk calculation.  It was only ever applied at a certain stage of the leasing process.  In a previous stage, the customer's address, bank account and other confirmatory details had been checked against their company name - in other words, they were who they said they were.   So this check was never required as part of the credit risk calculation, and now it is not part of the service.

Of course, all other applications also make such checks.  However, one application in particular - for debt consolidation - did not make the check itself.  It did not need to, since it implemented credit risk analysis via a call to ancient mainframe code.  This mainframe code was well-written for the time - so carefully written, in fact, that before carrying out any calculations at all it would do various checks, including of company details against their address, bank account, and so on.

At the time of creating the debt consolidation application, it was well-known that the mainframe application did such checks.  This is why they were not included in the debt consolidation application.  However, there is no longer anyone around that remembers this.  In fact, the only description of these generic checks is buried deep in the original program documentation for the mainframe application as a whole - no-one saw a need to repeat the description of the checks in the documentation of each individual module.  At the time of its own creation, the mainframe application was monolithic, after all.  So the documentation for the credit check module of the mainframe application only mentions the credit risk calculation itself.

The result of all this is that the new credit risk service appears to be a perfectly valid replacement for the legacy mainframe code.  Further, the debt consolidation application was actually the last remaining user of the legacy mainframe application, so the legacy application is discontinued on switching to use of the new internal credit risk service.  No-one has noticed that in the process, a baby has been thrown out with the bathwater.  Credit risk calculations in the debt consolidation application are now being made on companies that may not be who they say they are.  Anyone could use a company name that belongs to someone else to borrow money.

This may never happen.  But if it does, no-one will even notice.  Until, that is, debt consolidation customers start to vanish without trace, taking your organization's money along with them.

TAKE AWAY

I have often seen exactly this kind of thing happen as a result of refactoring.  In the simplest case, an extracted method may fail unpredictably on re-use due to lack of a null check enclosing its logic, since in the original code from which it was taken the null check was performed outside the lines selected for extraction.

However, until recently the side-effects of poorly implemented refactoring have generally been limited to within a single application.  Hence, such side-effects are often discovered in pre-release testing, and even if disaster hits post-release, it can be contained.

However, with SOA,  refactoring is no longer of components.  Refactoring is now of services.  Unintentional side-effects may have long-term, organization-wide consequences.  It has become much harder to discover problems such as the above - and once they are discovered, it is much harder to put things to rights.

So what is the solution?  In fact, the clue to it is contained in the scenario description above.  In future postings to this blog, I will show how to use the principles and patterns of Human Interaction Management to deal better with the extension of refactoring to an enterprise scope, along with the more general issues of impact analysis and the assessment of ROI in an SOA environment.

PS

On Friday I am talking at Javapolis, and will be at the conference from Tuesday evening onwards - if any readers of this blog also happen to be attending, and would like a chat, do come and find me.

Posted by keithhb in Service-Orientated Architecture | Permalink | Comments (0)

Many large organizations are now proudly rolling out their brand new SOA infrastructures.  Is yours?

Perhaps your company, like many others, is well on the way to transforming its integration backbone from an old-fashioned "point-to-point" scenario (in which each application must each integrate with each other application separately) to a forward-looking SOA approach (in which each application need only integrate with a single service layer).

In one case I came into contact with recently, the organization concerned referred to its service layer as the "Matrix" - a phrase which carries disturbing overtones of technology spinning out of control.  And indeed this, it seems to me, is exactly what is happening.

Let's consider a couple of very simple issues.

Impact analysis

How is your organization coping with change to services?

In fact, it is non-trivial even to define what is meant by "change" to a service.  How many of the following changes would generate an impact analysis in your organization:

• Addition of a parameter?


• A new possible value for a parameter?


• Change in algorithm?


• Switch of underpinning database/application?


• Use of underpinning databases/applications by another application?


• A new security check?


• New logging procedures?

The textbook answer is this:

A proposal for any such change would generate an organization-wide impact analysis, driven by logical analysis of pi-calculus bisimilarity between the old and new scenario.

However, only a very few organizations in the financial services, military and critical systems fields are following this route.  What is your company doing?  Many seem to be trusting to luck that no-one acccidentally breaks anything serious.

Cost-effectiveness

It is a truism that no software is static.  So services require continual attention if they are to stay useful.  And this costs time and money.

So how is your organization assessing:

• The business value-add provided by each service?


• The degree of maintenance effort appropriate for each service?


• Whether or not enhancements are necessary for each service?


• What actions to take when service-level agreements (SLAs) for a service are violated?


• Which IT staff to assign to which services?


• When to dispose of services?


• When to combine existing services, possibly with some loss of functionality?

Here there are not yet even any textbook answers.

TAKE AWAY

It has been generally accepted for some time that investment in IT is not, on its own, going to give your organization any strategic advantage:

In 2003, Chris Verhoef of Vrije University, in his keynote speech at the IEEE International Workshop on Source Code Analysis and Manipulation, claimed that there was little evidence of a positive correlation between IT investment and return on shareholders' investment.  He was drawing on his own studies, plus those by Paul Strassman (the ex-CIO of the US Department of Defense), who claimed that no relationship existed between IT investment and company profitability (P. Strassman and T. Pisello, IT Value Chain Management: Maximizing the ROI from IT Investments, Information Economics Press, 2003).
EQUITY and the Problem of Return on IT Investment

Strassman showed that you have to box clever if you want to get anything at all out of IT.  I don't see organizations boxing very clever with regard to SOA.

In the next postings to this blog, I will be considering how to use SOA to advantage - in particular, I will deal with the problems described above as well as others.

And I will suggest ways to deal with such issues without spending a fortune on yet more technology!  Rather, the answers, as so often, are right under our noses.

Posted by keithhb in Management • Service-Orientated Architecture | Permalink | Comments (0)