In my consultancy work I have seen many situations in which people who knew they could deliver real business value by utilizing open source tools found their efforts strangled by corporate insistence on lengthy evaluation processes. Apart from the time and effort involved in going through such processes, which in itself is a serious deterrent, the cost of carrying out such processes is so high that you might as well buy commercial software from a leading brand.

Consultants and lawyers are the ones who really love this kind of thing - it keeps them in work, and they can make a good case for such an approach to senior executives by stressing risks. In fact it is partly the FUD (Fear, Uncertainty and Doubt) generated in this way that holds back the adoption of open source tools. The criteria typically listed for evaluating open source tools are in fact equally applicable to commercial software, but usually not applied in anything like such force to organizations that have an "Inc" or "plc" in their name - and it's harder to anyway, not only since commercial companies are much less transparent than the open source community, but also because they may have less of an issue with honesty. Large manufacturing companies may think it normal to hire a business integrity consultant to check out a new parts supplier, but few CIOs do so with their software vendors.

The hazards of open source are not those typically those voiced as objections by consultants anyway – unstable or insecure software, availability of support, and legal issues. The open source projects I have discussed, for example, are perfectly viable from all these perspectives. In general, major open source software applications are written at least as well as leading commercial products (often by the same people), enthusiastically supported by expert and helpful developers (as opposed to knowledge-free call center staff), and transparently licensed (via industry-standard agreements). More to the point, anyone with enough experience in IT knows that leading, expensive commercial products are often deeply buggy, poorly supported and legally vulnerable.

So, how should one evaluate an open source system? Is there an easier way to determine the risk?

In my own software development work, I have used many different open source products, and always go through the same three-stage process when selecting one. First, I evaluate my own need as clearly as possible. What is it that I am really looking for? People often work under false assumptions when evaluating new software - assuming, for instance, that they need a database when all they really need is a means of saving objects to permanent storage, or that they need Web service tools that support the full WS-* stack when all they really need is a means of enabling communication between distributed components. Conversely, you may be looking for a tool to help write HTML or XML when you would be better served by a higher-level system that generated such low-level code automatically for you, concealing all the complex details.