*Editor’s Note: This is an excerpt prepared for ebizQ from a report originally issued by the Patricia Seybold Group on December 15, 2005. The full report details are available at http://www.psgroup.com/detail.aspx?ID=668

Read part I of the ebizQ excerpt here.

PLANNING THE VENDOR RUN-OFF

• Define Incremental Test Cases. Both the vendor run-off and your in-house tests will be time constrained. To maximize the time, define tests cases that can be built out incrementally, to demonstrate product capabilities. For example, start with a simple integration scenario that exposes a back-end customer database through a customer lookup service. Next, incorporate the customer lookup service in an orchestration, perhaps open customer account. Finally, add the open customer account orchestration to a process, such as loan approval, which could also include logging and notifications.
• Describe Your Test Cases in Business Terms. The test case requirements you send to the vendors should be described in terms of “what,” not “how.” That way, you can see and evaluate the vendor’s recommendation for “how.” For example, your description of the customer lookup case might be: The customer lookup service accepts customer ID, name or telephone number as input, and returns customer ID, name, address, phone, and status. The customer database is stored in an Oracle DBMS, and is accessible through a plain old Java object (POJO).
• Communicate Standards and Constraints. Make sure you share any in-house standards or constraints with the vendors. For example, if you have standardized on Restful (XML/HTTP) services rather than Web Services, articulate that as a requirement. If privacy compliance is important and certain data elements need to be stripped or encrypted, articulate that as a requirement.
• Request an Architecture Review. Based on your test cases, standards, and constraints, ask the vendor to provide a suggested architecture to resolve your tests. The architecture should include views of each test case (integration scenario) as well as the design, development, and runtime environments. Use the architecture review as a final gate before bringing a vendor on-site for the run-off. If the architecture presented doesn’t match your vision, and adjustments can’t be made, move on to another vendor.
• Negotiate Post Run-Off Resource Access and Ownership. In your deal, secure access to the software installation and run-off deliverables for the duration of your internal proof-of-concept. Be clear as to the ownership and rights in respect to the run-off deliverables, particularly software components that contain your IP.
• Be Prepared! Secure and prepare equipment and personnel before the vendor is on-site. Don’t allow small problems––security access, operating system patch levels, or incorrect personnel—to obstruct your test period, and provide the vendor a built-in excuse.
• Work One Product at a Time. Schedule one vendor at a time. Block a one-week test period (minimum) immediately following the vendor run-off. Capture all results, questions, concerns, and reactions before moving on to the next product.