Planning and Conducting Enterprise Service Bus Rides (Part II)

*Editor’s Note: This is an excerpt prepared for ebizQ from a report originally issued by the Patricia Seybold Group on December 15, 2005. The full report details are available at

Read part I of the ebizQ excerpt here.


  • Define Incremental Test Cases. Both the vendor run-off and your in-house tests will be time constrained. To maximize the time, define tests cases that can be built out incrementally, to demonstrate product capabilities. For example, start with a simple integration scenario that exposes a back-end customer database through a customer lookup service. Next, incorporate the customer lookup service in an orchestration, perhaps open customer account. Finally, add the open customer account orchestration to a process, such as loan approval, which could also include logging and notifications.
  • Describe Your Test Cases in Business Terms. The test case requirements you send to the vendors should be described in terms of “what,” not “how.” That way, you can see and evaluate the vendor’s recommendation for “how.” For example, your description of the customer lookup case might be: The customer lookup service accepts customer ID, name or telephone number as input, and returns customer ID, name, address, phone, and status. The customer database is stored in an Oracle DBMS, and is accessible through a plain old Java object (POJO).
  • Communicate Standards and Constraints. Make sure you share any in-house standards or constraints with the vendors. For example, if you have standardized on Restful (XML/HTTP) services rather than Web Services, articulate that as a requirement. If privacy compliance is important and certain data elements need to be stripped or encrypted, articulate that as a requirement.
  • Request an Architecture Review. Based on your test cases, standards, and constraints, ask the vendor to provide a suggested architecture to resolve your tests. The architecture should include views of each test case (integration scenario) as well as the design, development, and runtime environments. Use the architecture review as a final gate before bringing a vendor on-site for the run-off. If the architecture presented doesn’t match your vision, and adjustments can’t be made, move on to another vendor.
  • Negotiate Post Run-Off Resource Access and Ownership. In your deal, secure access to the software installation and run-off deliverables for the duration of your internal proof-of-concept. Be clear as to the ownership and rights in respect to the run-off deliverables, particularly software components that contain your IP.
  • Be Prepared! Secure and prepare equipment and personnel before the vendor is on-site. Don’t allow small problems––security access, operating system patch levels, or incorrect personnel—to obstruct your test period, and provide the vendor a built-in excuse.
  • Work One Product at a Time. Schedule one vendor at a time. Block a one-week test period (minimum) immediately following the vendor run-off. Capture all results, questions, concerns, and reactions before moving on to the next product.


  • Get Everyone on the Same Page. Day one, hour one, bring your team and the vendor team together to walk through the objectives, schedule, roles, and expectations for the vendor run-off period. Be explicit about the rules of interaction during the test period—particularly when it is appropriate for observers to ask questions, and how that time will be recorded. Remember, the vendor is in a race.
  • Learn by Observation. Take the opportunity to watch as the vendor team exercises the products in design, development, deployment, and runtime. Make note of the artifacts the vendor has produced on-site and what (if any) were produced ahead of time.
  • Schedule Check-Ins. Assuming the run-off is a multiday engagement, schedule start- and end-of-day check-ins. Discuss status, problems encountered, and the next day’s plan. Use this time to get a demo of work to date.
  • Use Problems as Opportunities. If there are problems during development or execution, use them to learn about the product’s problem detection and resolution capabilities.
  • Remember Quality of Service and Protection. While it is difficult to test all aspects of QOS and QOP in a run-off, ask the vendor to share how QOS and QOP can be achieved in your environment. The information may be relayed in a whiteboard discussion, through documentation, or pointing out console options.
  • Stay in the Room! Observe the entire process, to learn, and to evaluate.

As always, choose wisely, for your business.

About the Author

Brenda Michelson is an IT strategist, hands-on practitioner, and the voice of business-driven architecture. Brenda’s day job is principal consultant for Elemental Links. Brenda also contributes architecturally to the Patricia Seybold Group. Brenda spent 19 years in corporate IT, most recently as Chief Enterprise Architect for L.L. Bean. At L.L. Bean, Brenda was responsible for the articulation and execution of the enterprise architecture strategy (J2EE transformation, enterprise integration, SOA and EDA), strategic planning, portfolio management and talent development. Previous to L.L. Bean, over the span of 10 years, Brenda provided development services for Insurance, Banking, a Chip Manufacturer and a world leader in Aircraft Engine Design & Manufacturing. Email Brenda.

More by Brenda M. Michelson

About Elemental Links

Elemental Links is an IT consulting and advisory practice specializing in strategy, architecture, and portfolio planning for business-driven IT.