Selecting A Message Broker

In his next two columns, our EAI expert David Linthicum provides an easy-to-understand guide for selecting a message broker.



If there is one single type of middleware that best addresses the EAI (Enterprise Application Integration) problem, I would have to say that it is message brokers. Message brokers are really just another layer on top of traditional messaging systems, such as JMS or MQSeries, allowing EAI architects or developers to programmatically reformat and route information, moving from one system to another. This type of technology minimizes the impact on both the source and target systems, thus reducing the cost of EAI and maximizing flexibility. Message brokers place a processing layer between the disparate applications and platforms, accounting for the differences in application semantics, design patterns, data formats, and operating systems within the message broker (see figure 1).

Figure 1: "Message brokers place a processing layer between the disparate applications and platforms, accounting for the differences in application semantics, design patterns, data formats, and native features of the platforms."

Selecting the right message broker for the job is a daunting task for middleware vendors, new and old, who are moving into the world of message brokers. Message broker vendors are over-hyping the features and functions of their products, and buyers have few data points to compare and contrast available products, because the technology is so new. What is more, other middleware products, such as application servers and distributed objects, are laying claim to the EAI space and are promoting their products as being mutually exclusive to message brokers. The fact is that each solves very different problems, and each has its place within EAI.

To understand what to look for in a message broker, you have to comprehend the components and features that this new technology has to offer. By doing this youíre able to establish your own set of criteria for your specific problem. Things to look at include messaging, adapters, user interface, platform support, transformation, routing, rules processing, scalability, security and management.

Messaging

Since message brokers use messaging systems to move information between systems, the message transport layer that the message broker supports or provides is of great interest. Most message brokers support some sort of asynchronous transport layer, either proprietary or open. For instance, many message brokers can leverage IBMís MQSeries message queuing software, or more open systems such as the Java Messaging Service (JMS) from JavaSoft, or the Object Messaging System (OMS) from the Object Management Group. Some message brokers offer the ability to select any number of messaging systems, even to mix and match messaging systems to meet the requirements of the problem domain.

The key thing to look for is industry support for a particular messaging layer, such as market penetration, as well as the number of platforms that the messaging layer runs across. Itís also important to consider the openness of the messaging layer; for instance JMS is an open messaging standard supported by a number of vendors, as is OMS. Donít forget to consider performance of the messaging layer and support by third party products.

Message brokers typically ship with their own messaging systems built in, but they tend to be proprietary. Architects should not undervalue the importance of the messaging layer, since it can limit the extensibility and flexibility of the EAI solution. For example, itís going to be difficult to adapt a proprietary messaging layer to other messaging systems that may appear over the years in an enterprise.

Adapters

The notion of adapters for message brokers has been considered for years. However, as with so many other things, each message broker vendor has had its own concept of exactly what an adapter should be. To date, there is no standard.

Adapters are a layer between the message broker and the source or target application or data. This could, for example, be a set of "libraries" that map the differences between two distinct interfaces -- the message broker interface and the native interface of the source or target application -- that hide the complexities of that interface from the end user or even the EAI developer using the message broker.

For example, a message broker vendor may have adapters for several different source and target applications (such as SAP, Baan, PeopleSoft, etc.), or adapters for certain types of databases (such as Oracle, Sybase, or DB2), or even adapters for specific brands of middleware. There are two types of adapters in the world of message brokers: thin adapters and thick adapters. Adapters may behave in two different ways: dynamic and static.

Thin adapters are offered by the most popular message brokers today. In most cases, they are simple API wrappers, or binders, delivered as a set of libraries, that map the interface of the source or target system to a common interface supported by the message broker. In other words, they simply perform an API binding trick, binding one API to another.

Thin adapters have the advantage of simplicity in implementation. Since there is no additional "thick" layer of software between source and target applications, there is greater granular control. They have a number of disadvantages, though. Since nothing more than trading one interface software for another is being accomplished, thin adapters impact performance without increasing functionality. Quite a bit of programming will still be required. Complicating matters is the fact that common APIs which are being mapped are almost always proprietary.

Thick adapters, in contrast, provide a lot of software and functionality between the message broker infrastructure and the source or target applications. The thick adapterís layer of abstraction makes managing the movement or invoking processes painless. Since the abstraction layer and the manager handle the difference between all the applications requiring integration, there is almost no need for programming.

Thick interfaces are capable of accomplishing this, because they provide a layer of sophisticated software that hides the complexities of the source and target application interfaces from the message broker user. The user sees only a business-like representation of the process and the metadata information as managed by the abstraction layer and the adapter. In many cases, the user connects many systems through this abstraction layer and through the graphical user interface, without ever having to resort to hand coding.

Repositories play a major role in the thick adapter scenario. The repository is able to understand much of the information about the source and target applications, and to use that information as a mechanism to interact with those applications on behalf of the message broker.

Moreover, several abstraction layers may be created around the types of applications to be integrated. For example, there may be an abstraction for common middleware services (such as distributed objects, message-oriented middleware, and transactional middleware). There may also be an abstraction layer for packaged applications; another layer addresses the integration of relational and non-relational databases. However, the complexities of the interfaces that each entity (middleware, packaged applications, and databases) employs are hidden from the end user.

In addition to the thick versus thin debate, there is also the notion of static versus dynamic adapters. Static adapters, the most common today, have to be manually coded with whatís contained within the source and the target systems. For example, static adapters have no mechanism for understanding anything about the schema of connected databases, and therefore must be configured by hand to receive information from the data source. Whatís more, if the connected database schema changes, static adapters have no mechanism for updating their configuration with the new schema. This means that the system will probably fail.

In contrast, dynamic adapters are able to "learn" about the source or target systems connected to them, through a discovery process that the adapters go through when first connected to the application or data source. This discovery process typically involves reading the database schema, information from the repository, or perhaps the source code, to determine structure, content, and application semantics of the connected system. In addition to simply learning about the connected system, dynamic adapters are also able to relearn if something changes within the connected system over time. For instance, changing the customer number attribute name will automatically be understood by a dynamic adapter.

Clearly itís better to select a message broker with both thick and dynamic adapters. However, the state-of-the-art has yet to address both, and message broker vendors have some work to do before their adapters are able to live up to the hype. For now, your best bet is to look carefully at the information about available adapters, such as thick or thin characteristics, and place a few in use within a pilot project. Itís always a good idea to make vendors earn your business.

In addition to selecting off-the-shelf adapters, your message broker of choice needs to provide you with the ability to build custom adapters for less popular systems, including "one-off" custom systems. This means providing a development kit, common libraries, and hopefully documentation and training, allowing developers to create adapters.

David S. Linthicum is the CTO of SAGA Software and internationally known EAI expert. David has authored over 300 articles for PC Magazine, InfoWeek, BYTE, DBMS Magazine, Database Programming and Design, Object Magazine, Component Strategies, and Application Development Trends. He has authored or co-authored six books including his latest David Linthicum's Guide to Client/Server and Intranet Development, published in June 1997, and Enterprise Application Integration out this year. He is a frequent speaker at key technology forums and conferences throughout the United States.

About the Author

David S. Linthicum (Dave) knows cloud computing and Service Oriented Architecture (SOA). He is an internationally recognized industry expert and thought leader, and the author and coauthor of 13 books on computing, including the best selling Enterprise Application Integration (Addison Wesley). Dave keynotes at many leading technology conferences on cloud computing, SOA, Web 2.0, and enterprise architecture, and has appeared on a number of TV and radio shows as a computing expert. He is a blogger for InfoWorld, Intelligent Enterprise, and eBizq.net, covering SOA and enterprise computing topics. Dave also has columns in Government Computer News, Cloud Computing Journal, SOA Journal, Align Journal, and is the editor of Virtualization Journal.

In his career, Dave has formed or enhanced many of the ideas behind modern distributed computing including Enterprise Application Integration, B2B Application Integration, and SOA, approaches and technologies in wide use today. For the last 10 years, Dave has focused on the technology and strategies around cloud computing, and how to make cloud computing work for the modern enterprise. This includes work with several cloud computing startups.

Daveís industry experience includes tenure as CTO and CEO of several successful software companies, and upper-level management positions in Fortune 100 companies. In addition, he was an associate professor of computer science for eight years, and continues to lecture at major technical colleges and universities including the University of Virginia, Arizona State University, and the University of Wisconsin.

More by David S. Linthicum