The Metrics of CX, Part 1: Traditional CX Measurements and Their Challenges

The following is based on ideas from my recent book, Meaningful Measurement of the Customer Experience, now available everywhere. 

It can be challenging to meaningfully measure customer experience in any organization. Several metrics have been touted as good ways to see the effects of CX and its impact on customer satisfaction, loyalty, and recommendations. In this chapter we’re going to review some of these, though there are many we won’t get the opportunity to explore.

Chances are, if you are either a customer experience professional or a consumer—and chances are you are at least one of these!—you have run across them in one form or another. We will discuss these and then talk about some of the shortcomings of one of the most popular ones, the Net Promoter Score, as we pave the way to introduce the Center of Experience framework for CX measurement.

CSAT

Short for customer satisfaction score, CSAT is often used as a KPI to measure exactly what you’d expect: customer service, product quality, and other related metrics in many different types of organizations. The score itself is a percentage from 0% (the worst possible score) to 100% (the best).

CSAT is measured via a survey that has respondents answer a question similar to the following: “How would you rate your overall satisfaction with the [product or service] you receive?”

The answer is provided using a scale from 1 to 5, similar to the following:

1. Very Unsatisfied
2. Unsatisfied
3. Neutral
4. Satisfied
5. Very Satisfied

 These scores are then compiled into that percentage score from 0-100%. You’ve undoubtedly seen and experienced one of these as a consumer, even if you haven’t been part of administering one as a customer service or customer experience professional.

Customer Effort Score (CES)

This is an interesting method of measuring customer experience, based on the premise supported by some 2010 research by CEB that improving an individual’s ability to smoothly get an issue resolved can result in higher degrees of loyalty than focusing effort and resources attempting to delight them[ii]. In other words, focus on helping your customers quickly and easily get back to their lives by reducing friction in your processes and interactions, and they will reward you as repeat customers.

While CES offers a unique view and perspective, it is best to use this measurement with others in order to provide a clear picture of customer experience and satisfaction.

Net Promoter Score (NPS)

One of the more popular measurements of customer satisfaction is the Net Promoter Score, often referred to by its initials as NPS.  While it has been touted by some as a singular metric to determine if your customer experience initiatives are successful, it has some significant shortcomings if it is the primary measurement for such things, and some experts are casting doubt[iii] on using this statistic without caveats. In this chapter, I’ll explore why NPS, valuable as it can be, should not be the only method of measurement for CX.

Net Promoter Score was developed and trademarked by Bain & Company, Inc., Satmetrix Systems, Inc., and Fred Reichheld in 2003. The score is based on the question “How likely is it that you would recommend our company/product/service to a friend or colleague?” rated on a 0-10 scale.

Those that answer are broken into 3 groups:

• Promoters, or those who answered 9 or 10
• Passives, or those who answered 7 or 8
• Detractors, or those who answered 0-6

 It’s as simple as that, and therein lie some of the issues with using Net Promoter Score as your major method of measuring customer experience.

That said, research by the London School of Economics has shown that companies with an average NPS increase of 7% have been able to correlate an average 1% growth in revenue[iv]. 

Challenges with using NPS and other single Customer Satisfaction Metrics.

I’m going to focus primarily on NPS, but my intent is not to “pick on” a single metric or approach. The goal here is to highlight challenges of why relying on a single metric can prove problematic.

The biggest issue with using Net Promoter Score as your primary or singular customer experience measurement is that it doesn’t take a lot of other factors into account. Let’s explore some of these below.

NPS IS NOT A GOOD PREDICTOR OF GROWTH OR FUTURE BEHAVIOR

The question posed in the Net Promoter Score is to answer whether you would be willing to recommend a product or service to a friend or colleague. It doesn’t ask if you would be interested in buying the same or similar product yourself, or even if you are happy with your purchase. A lot is assumed within that question.

Also, many times people say one thing and do another, sometimes skewed by if they feel their behavior is being too closely observed, and sometimes they are simply happy that they completed their purchase and haven’t actually even had time to use the product or service yet.

NPS ONLY SHOWS A SINGLE DIMENSION OF LOYALTY

Companies that rely heavily on NPS as their metric of success are also only seeing one dimension. While likelihood to recommend is a good thing to know, there are other ways you should look at your customer experience as well.

For instance, Business Over Broadway identified three different types of loyalty: retention loyalty, advocacy loyalty (which NPS fits into), and purchasing loyalty. Net Promoter Score really only takes the second type into account. Solely using NPS means you’re missing out on understanding the other two types.

NPS TENDS TO SKEW TO THE EXTREMES

If you’re like me, you’ve most likely run into the phenomenon of people falling into either extreme when it comes to giving feedback to companies. Either they are vocal advocates, or complaining about the service. Rarely do you have someone reply to say that they are only moderately happy.

For instance, in the heat of the moment, someone is usually very happy or very dissatisfied with their experience. Often someone who feels at a “5” is not going to bother providing feedback at all. Therefore, you are more likely to get feedback from those at the extremes than in the middle of the spectrum.

Additionally, a so-called “detractor,” or someone who rates 0-6 on the Net Promoter Score scale, is not necessarily someone that is actively bad-mouthing your company or product. They may even be willing to give you another chance but had a single bad experience. Without contextual information and surrounding data, it’s difficult to rely on that single number.

NPS DOESN’T TELL YOU WHAT YOU NEED TO FIX

Finally, the Net Promoter Score is only an indicator of how likely a customer is to recommend you in the moment. If they give you a bad score, it’s unclear exactly why they did, so it’s difficult to understand how you can improve the situation in the future. Likewise, if they give you a great score, it may be unclear if the entire customer experience was rewarding and something the customer would recommend, or if there were particular high points.

In my experience, asking for a little more detail can go a long way in understanding NPS ratings. Also, having diagnostic analytics throughout your sales and product usage process that are more objective can also feed you information that can help you understand what might be happening. First instance, if the time to complete an e-commerce transaction suddenly doubles because of server errors, and you see your NPS scores plummet, you can very easily correlate these two things. While there may not always be such an obvious connection, using diagnostic metrics can help you fill in gaps more easily.

Don’t get me wrong. Net Promoter Score is not all bad, but it isn’t an end-all be-all to solve your customer experience challenges. You can use NPS as one element in your array of customer experience measurements as long as you take the other factors discussed above into account. 

In my professional experience advising companies in their customer experience approach, I have always tried to use NPS (or Employee Net Promoter Score for employee experience initiatives) as part of a larger set of measurements in order to provide the most meaningful and well-rounded approach to measuring CX.

Having said that … a caveat

I will give the caveat here that I think NPS, CSAT or other measurements can play a supporting role, and their ability to provide a consistent baseline in order to measure overall progress year over year can be incredibly beneficial. I say the same thing about employee engagement scores to organizations I work with on employee experience initiatives. Those engagement scores alone are not enough to measure the effectiveness of individual efforts or event individuals, but in the aggregate, they can show effects of changes made within the organization over time. These “snapshots” of progress can indeed be incredibly helpful and valuable, but they don’t substitute for a more holistic approach, which we will explore in the chapters that follow.

In the next article we’re going to start a discussion of a customer experience measurement framework I propose you consider. It takes into account the full range of CX metrics and can provide a more holistic view into the customer experience. 

This article is based on ideas from my recent book, Meaningful Measurement of the Customer Experience, now available everywhere.