Choosing Metrics for Customer Exerience Management

I spent some time yesterday researching metrics for customer experience. This was far from idle curiosity, since one of our premises at Client X Client is that businesses should focus on the single measure of customer lifetime value. I wanted to see how other people approach the problem.

Our position turns out to be unusual. Of course, many people mention lifetime value, but usually as one in a long list of measures. I’ll come back later to why we disagree, but first let’s look at what other people propose. I found the measures fall into several basic categories:

- operational measures such as Web site response time, call center hold time, number of upsell offers made, number of emails sent, and so on. These measure company-controlled activities and the quality of the experience provided to the customer. Mystery shopper scores and other pure “experience” measures would also fall into this category, since they measure what the company does rather than how customers respond.

- behavior measures such as store visit length, shopping cart abandonment rates, complaint rates, and call center hold queue abandonment rates. These measure customer activities and are often closely related to the company’s operational performance.

- results measures such as conversion rates, attrition rates, and share of wallet. Like behavior measures, these measure customer activities. But they have direct financial values and thus can be tied into a lifetime value calculation.

- attitudinal measures such as customer satisfaction scores, net promoter scores, and customer comments. These are indirect rather than direct measures, and as such must be treated carefully because people often say one thing and do another. But they are useful because they can summarize the consequences of the many different experiences that customers receive.

It’s easy to say that all these measures are important. But it’s not so easy to collect and interpret them all. In fact, it’s overwhelmingly difficult. Down at the tactical level—which is where businesses make and lose money—you will need to focus on specific operational and behavior measures for specific projects such as Web site optimization or IVR deployment. In addition, you should continuously track at least a few operational and behavioral measures as leading indicators of business results. But if you try to track too many of them, they just become background noise.

The trick is to place the operational, behavioral and attitudinal measures in context by linking them to results. This is where we get back to lifetime value: it is the one ultimate result. All other result measures are just contributors to lifetime value.

Think about it. Knowing the relationship between, say, Web site load times (an operational measure) and conversion rates (a results measure) may let me predict the impact on conversion of an investment that would improve response time, or the impact of letting response time deteriorate. But to understand what the change in conversion rates is worth, I have to relate it to a dollar measure—that is, to lifetime value. (Yes I could use a simpler measure, like revenue. But I really want to consider the other implications such as the impact of conversion rates on long-term retention rates.)

The benefit of relating everything to lifetime value is that you can now compare wildly different decisions: do I upgrade the Web site or install that new IVR? Knowing that one will improve conversion rates by 5% while the other will cut hold times by 10% isn’t particularly helpful. Knowing that one will change lifetime value by $1 million and the other by $2 million clarifies matters quite nicely. (Obviously we’re talking about the aggregate lifetime value of all customers—a topic with nuances of its own.)

Similarly, at a tactical level, if I’m watching trends in a bunch of operational measures and several start to falter, knowing the impact of each change on lifetime value lets me determine which to address first. This is how you avoid being overwhelmed by too many operational and behavior measures: capture the trends on as many as you like, but only highlight the ones with a substantial business impact.

Now you see why Client X Client considers lifetime value so important. But if we’re right, the implication is you need a lifetime value forecasting model with all the underlying formulas that connect changes in other measures to the lifetime value figure. This type of model isn’t easy to build. The need for it is the reason I spend so much time looking at things like marketing mix models, agent based models and multivariate testing systems. From what I see, the techniques to build the lifetime value forecast models are indeed available even though they are rarely used in quite this way. Yet I would argue that customer experience management can only succeed as an enterprise management tool if it has a meaningful measurement regime. So building these models is a critical requirement for CEM’s future development.