Click here to send Jon Bachman a question -- he'll respond in this space.

David Kelly: Welcome to a special eBizQ podcast devoted to ESBs and integration patterns. I'm your host, eBizQ senior analyst, David A. Kelly. With us is Progress Software's senior director for product marketing, Jon Bachman. Welcome, Jon! Thanks for taking the time to talk with us today.

Jon Bachman: Oh, thank you, David. I look forward to this session.

DK: Great! One of the things I wanted to start off with is over the past few years, Enterprise Service Bus, or ESBs, have emerged as a key enabler for more agile and effective integration across the enterprise systems and business processes. Yet, business success is more than simply purchasing the right technology. It often means knowing how to use what you have most effectively. Can you please give our listeners a perspective on the key best practices they should be thinking about when it comes to ESBs?

JB: Yeah, certainly. We have found from our customers that the first project that most people use an ESB for is a relatively simple one that has to do onloading and offloading files, moving them from batch processing styles of sequential, overnight processes to a more real-time continuous pipeline of processes. And, so I guess one of the key practices here is: start with a project which is easily understandable, one which will benefit the organization when it is complete, and don't be surprised if the first project is a relatively simple replacement of an FTP or file transfer type application where you had two to three different file transfers combined with some sequential processing.

It is the most common first project for an ESB. And we call that pattern the continuous pipeline processing pattern. And the primary requirements are to understand how the product or the ESB you've purchased supports what I would call file on-ramping and file off-ramping. And the ability to break a batch of transactions that are sort of encoded in that batch file into individual messages or transactions that are then turned into events on that enterprise backbone or Enterprise Service Bus.

DK: Okay. You mentioned integration patterns as one of the parts of the solution. Can you provide some additional insight into what you mean by integration patterns and how they work with ESBs?

JB: Yeah, sure. The patterns that I just mentioned, which had to do with sort of file to sort of real-time processing is one of the patterns that we see frequently from amongst our customers. There are two or three others that I think we should talk about. One of which, is access to remote data. If you think about a portal as an example, it's very common for a portal to assemble data from multiple data sources into a single page and present that to a user. But when you get into larger scale situations where the data that's trying to be accessed is actually not local to the application that's serving the pages, you have an integration problem that really we talk about as the remote information access problem, where you have to orchestrate the request and the reply to those multiple back-end data sources and assemble the response back for the end-user.

There are other patterns which are similar to that which are repeatable and frequently found in enterprise integration and SOA projects in general. Another one is remote data distribution. You might have, for example, an application that is the master record keeper for customers or products. And when a change is made to a customer or to a product definition, you need ten or twenty other applications to know about that as quickly as possible. You want to make sure that you reliably distribute that information and that change out to those applications without failure.

And then finally, a fourth one, which I think is useful to imagine is it's a combination of the remote information and access to remote data distribution, where you have applications that are relying, in real-time, on the change in status of some enterprise event. So, for example, airports have a lot of applications that run based on the arrival or departure of an aircraft at the terminal or gate. And you need to set in motion, concurrent processing to respond to that real-world business event.

So, just to summarize, there are patterns in enterprise integration whether it's continuous pipeline processing access or distribution of data, or the response to real-time business events that are very commonly reoccurring in enterprises and can actually be turned into patterns or templates that can be reused at a very high level of abstraction.

And what that really allows you to do, is not worry about having to reinvent or do each integration once and then throw all your expertise away. You can actually capture that pattern and save it, both at the low-level components as well as the higher level, the macro level pattern itself. And then you form really a language, with which your team can talk with one another and a set of component definitions, event definitions, service interfaces that come as a package and allow you to rapidly deploy another instance with perhaps minor variations to that same pattern.

So, really, I think the key benefit of an ESB is that it provides a palette on which you can build enterprise integrations without worrying about sort of the scope of the project or how many different sites might be involved, how many different applications might be involved. And once you have that clear palette on which to paint, you can actually pull out the pattern and apply it to a particular business problem and solve it quickly and in a reproducible fashion.