Industry Solutions 

Topic Description

The Danger of Placebo Metrics

By John Stelzer, Director of Industry Development, Sterling Commerce , 02/14/2006

Print this article Email this article Talk Back! Write to Editor

When I was fairly young, my father taught me a lesson that has proven valuable in business to this day. Although I was years away from driving age, I started to show an initial curiosity about cars. Recognizing my new interest, my dad took me with him to the local garage to pick up the family car after a tune-up.

I remember being shocked by the $25 bill for the service. [On my meager 25-cents-a-week allowance, this seemed like an expenditure that would require a lifetime of saving to be able to afford.] When I commented on the “mammoth” tab, my dad pointed out that the tune-up was essentially free because the money he would save from improved gas mileage would more than cover the cost of the service.

He explained that before the service, we were getting about 16 miles per gallon, and that it typically improved to 21 miles per gallon after a tune-up. Gasoline was selling at the exorbitant price of 29 cents per gallon in those days. [Yes, there really was a time when you could buy gasoline for only 29 cents a gallon; and yes, I’m old enough to remember it; and no, it wasn’t as far back as the old hand-cranked models, either!] My father went on to explain that given the number of miles we would likely drive before the next tune-up, we would save much more than the $25 we had just spent for the tune-up.

In those days, we drove the car about 10,000 miles a year. At 21 miles to the gallon, that required roughly 476 gallons of gas. At 16 miles per gallon, it would have required 625 gallons. The difference (149 gallons) would cost $43.21 at 29 cents a gallon. This was clearly more than the $25 he had just paid for the tune-up.

This marked the beginning of my understanding of several important concepts. First, I was going to have to earn a lot more than 25 cents a week if I was ever going to own my own car—even if tune-ups did eventually pay for themselves! Second, if you’re going to understand the impact of a change, you’ve got to take the appropriate measurements before and after the change. And third, if you measure the wrong things—even if you do it before and after the change—you won’t know the actual impact of the change.