We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.
Start a Discussion
SOA
user-pic

What Are the Most Meaningful Metrics to Assess an SOA Solution?

Vote 1 Vote
Taken from last week's discussion on what's the percentage of reuse a company can expect from SOA, Kelly Emo posed this question: What are the most meaningful metrics to use to assess the success of a SOA effort, from the perspective of the IT teams? from the perspective of the business?

7 Replies

| Add a Reply
  • At the risk of being completely slaughtered as the day progresses, I'll go first...

    SOA is often touted as the "silver bullet" that will increase resusability, enhance maintainability, reduce rework, and so on. In that spirit, I think a metric of prime importance should be the Total Cost of Ownership (TCO) per Unit of Software Complexity (or Size) such as Function Points, Use Case Points, SLOC, etc. Assuming that you are already well down the path of SOA, a necessary first step would be to establish a historical baseline (pre-SOA) to compare against. The hope is to see current (SOA-based) measurements below the baseline with a continuous downward sloping trend.

  • I'd suggest a few metrics:


    • Mean time to service delivery: How quickly are new services being built introduced; a function of reuse

    • Cost of service delivery: both a function of reuse/development time and whether moving to a SOA model has reduced the overall cost of development

    • Rate of service use: How often is each individual service being used by the business. The idea is to evaluate whether development resources are being used to create services that are actually being used by the organization. A variant is rate of adoption which would look at how quickly, broadly the the service is adopted across the organization.

    • Marginal revenue per service: Looks at marginal or incremental revenue generated for the organization by each service. The idea is to highlight the value created for the organization as a result of the service. As an example, a service that allows a client to evaluate the risk of a complex financial instrument is inherently more valuable (for the user and consequently should generate more revenue for the company) than invoking Google Maps.

    That's all I got.

    Glenn

  • I think the key must be the number of times a service is reused. I would expect the higher level business services could potentially be used a low number of times and offer value but how many businesses have just one channel to market ? At the same time, if different 'similar' services are being used to run a business, I would argue this is very much a negative so this is a metric that must also be used. If the business cannot define and maintain unique services for each element of their business to service multiple business channels, they run the risk of getting different results depending on the channel.

  • I like Glen Gruber's suggestions. The last one, "marginal revenue per service," is probably the most important in the long run. It's all about how much is being delivered the business, period.

    I also see votes for rate of service reuse in the comments, though the value of "reuse" in SOA was beaten to a bloody pulp in last week's forum. I'll invoke the tree-fall-in-a-forest conundrum here: If you set up a catalog of SOA-aware services, but nobody uses them, do you still have a service-oriented architecture?

  • I think that there is no universal measure applicable to all enterprises. The following list includes some significant metrics:
    1. Agility which is translated practically to ease of making changes i.e. less Time and less resources compared to equivalent changes in pre-SOA environment.
    2. Service and Process Reuse from different Channels. Especially important is Reuse by Self Service Channels consumers. Self service channels usage cost could be compared to other channel usage costs per transaction.
    3. Service Reuse by multiple Business Lines applications.
    4. Shorter Time to Market of new products

  • I'm very late to this party. The one thing I haven't heard in this dialog is any discussion about having different success metrics for different stakeholders. I think a lot of the metrics above are meaningful to a VP of applications or business owner, especially the shorter time to market of new service/product or agility. Variants of this depending on the business goal could be metrics such as shorter time to on-board a partner, shorter time to deliver meaningful information, increased customer stickyness or satisfaction, increased percentage of cross-sell or upsell for commerce applications and so forth.

    That said, I also think there is a set of meaningful SOA metrics for other IT stakeholders. For shared services, how about % code re-use, decreased time to integrate, decreased time to migrate to new versions, or other key technical metrics?

    SOA can have a multi-faceted impact beneficially across IT and the business as long as the goals are well constructed and measured. Re-use is one aspect, but not the only...

Add a Reply

Recently Commented On

Monthly Archives

Blogs

ADVERTISEMENT