How Do You Do QA for Perfomance Management?

Doing performance management correctly is like trying to get my kids to wash their hands. Sure, I can create a process or rule for it: When our hands are dirty, we wash them. Then I can ensure that the components of the process are available and responsive: There's soap by the sink and the hot and cold water turn on easily. But even when I've set everything up, and have the metrics established (we need to wash our hands daily!), how can I be sure that they'll be achieved? And if they are achieved, are they being achieved correctly?

For example, it's pretty easy to identify when my older son isn't meeting our performance goals: His hands are clearly filthy. But it's a bit tougher with my younger son: He did wash his hands. But did he use hot water? Did he use soap? Did he actually scrub for a reasonable period of time (or did he simply pass them under the water)? It can be hard to tell. I've also discovered that there's usually no adequate feedback mechanism to help me identify when there are user or system problems. For example, we can be out of soap for days in their bathroom and I'll never hear about it. Or they may decide on their own to use one of those liquid hand sanitizers instead of actually washing their hands. It's hard to know sometimes.

The point is, even with the right process (or application) and tools, the effectiveness of the result really depends on actions of the users. If there are user failures (like failing to report missing soap, using cold water, etc.), then the resulting process is going to be ineffective.

As I will explore in my next column, it's the same with enterprise performance management and an organization's applications. When enterprise applications like SAP, Oracle or Seibel aren't being used effectively and efficiently by users, the business results will suffer.

One solution to this problem is putting solutions in place that can collect and manage information not only on your applications, but on the user experience of the users using those applications. Let's look at what that means.

Of course, you'd want to start by collecting traditional application management-related information like response time and availability. You'd also want to know the transactional response times-for example, how long does it take the application to respond after the "enter" key is pressed? But-and here's where it starts to go beyond traditional management solutions-you'd also want to know how long it takes for users to navigate through the application screens. Are they spending too much time on difficult to navigate or complete screens? Of course, you'd also want to collect information on which applications (and transactions) what people are using. It would also be helpful to collect information about quality of the users' experience and any errors in the infrastructure or application, whether they're cause by the user or by something in the IT environment. For example, perhaps users are getting errors trying to connect to certain applications or databases, but such errors aren't getting reported.

For many organizations, these types of metrics go well beyond what they have traditionally collected, analyzed or thought about. However, understanding this type of information can pay big dividends. For example, the help desk can use them to reduce the average time spent with each user, improve the coverage of the support staff, accelerate time to resolution and much more. But such information could also be used to tailor training on specific problem areas/screens/functions or application steps, reducing training time while increasing overall effectiveness. Or the information could be used by the line-of-business manager to pinpoint user adoption issues, minimize business disruption issues from application upgrades or new application adoption or proactively identify user efficiency or process errors. Lastly, such information can also help the IT organization through faster identification of performance issues.

If only I could put a system in place that will track the effectiveness of my family's hygiene activities.


About the Author

David Kelly - With twenty years at the cutting edge of enterprise infrastructure, David A. Kelly is ebizQ's Community Manager for Optimizing Business/IT Management. This category includes IT governance, SOA governance,and compliance, risk management, ITIL, business service management,registries and more.

As Community Manager, David will blog and podcast to keep the ebizQ community fully informed on all the important news and breakthroughs relevant to enterprise governance. David will also be responsible for publishing press releases, taking briefings, and overseeing vendor submitted feature articles to run on ebizQ. In addition, each week, David will compile the week's most important news and views in a newsletter emailed out to ebizQ's ever-growing Governance community. David Kelly is ideally suited to be ebizQ's Governing the Infrastructure Community Manager as he has been involved with application development, project management, and product development for over twenty years. As a technology and business analyst, David has been researching, writing and speaking on governance-related topics for over a decade.

David is an expert in Web services, application development, and enterprise infrastructures. As the former Senior VP of Analyst Services at Hurwitz Group, he has extensive experience in translating the implications of new application development, deployment, and management technologies into practical recommendations for enterprise customers. He's written articles for Computerworld, Software Magazine, the New York Times, and other publications, and spoken at conferences such as Comdex, Software Development, and Internet World. With expertise ranging from application development to enterprise management to integration/B2B services to IP networking and VPNs, Kelly can help companies profit from the diversity of a changing technology landscape.

More by David A. Kelly

About ebizQ

ebizQ is the insiderís guide to next-generation business process management. We offer a growing collection of independent editorial articles on BPM trends, issues, challenges and solutions, all targeted to business and IT BPM professionals.

We cover BPM standards, governance, technology and continuous process improvement, as well as process discovery, modeling, simulation and optimization, among many other areas. We follow case management, decision management, business rules management, operational intelligence, complex event processing and other related topics. We closely track important trends such as the rise of social BPM, mobile BPM and BPM in the cloud. We also explore BPMís use in functional areas, such as supply chain and customer management, and in key verticals, such as financial services, health care, insurance and government.

ebizQ's other BPM-oriented content includes podcasts, webcasts, webinars, white papers, a variety of expert blogs, a lively online forum and much more.