Assessing CX Using Multiple Data Sources

Assessing CX Using Multiple Data Sources

INSIGHTS
Assessing CX Using Multiple Data Sources
BRIEF

Assessing CX Using Multiple
Data Sources

New approaches to CX analytics in the world of expanding data and analytics.

As business leaders, employees and customers, we sense accelerating change everywhere. How can an organization ensure it is on the right trajectory for success? And where does CX fit in?

 

From then to now

A century ago, CX depended on informal relationships between store owners and their customers. In the 1950s, concepts around customer satisfaction began to emerge. Organizations measured satisfaction using mail surveys. They developed national and segment-specific indices by the 1970s. Other indices emerged to measure customer loyalty.

In the current century, CX Management became a core concept. The modes for gathering and measuring customer perceptions continued to evolve. We moved from mail surveys to mail and phone. We then shifted to email surveys, and in the last decade to survey by mobile phone, web, and social media.

We gather CX data more frequently too, especially for interaction- or event-based feedback. Automation, use of robots, and IoT devices enable us to track events, transactions and back end operations. The result is an astronomical growth of data. The emergence of new data sources is timely, because survey response rates have been declining, making them less representative of customer experience.

‘Big Data’ now dissects customer experience.

What’s left as a basis for competition is to execute your business with maximum efficiency and effectiveness, and to make the smartest decisions possible. And analytical competitors wring every last drop of value from business processes and key decisions.

– THOMAS H. DAVENPORT AND JEANNE G. HARRIS, HARVARD BUSINESS SCHOOL PRESS

Sophisticated analytics seek (1) to optimize business processes and (2) to make the best business decisions.

Tapping into sources of customer feedback – solicited or unsolicited – and customer behaviors lets us evaluate holistically. We can consider all organizational dimensions against a background of changing market forces. The data may be behavioral (operational, website), attitudinal (survey, social media, blogs, etc.), market, and financial. Execs can use it to grasp customer wants, product / service delivery relative to expectations, operational process efficiencies, and so on.

The many dimensions of an organization are interconnected and interdependent. We must analyze them that way. We must also go beyond linear relationships to the complex, intertwined fabric of the business. Specifically in relation to CX, customers expect consistency across all interactions. Leaders must implement actions that reflect the brand consistently across all channels and experiences.

Microsoft is a great example. Changes in business culture led to changes in business decisions. Microsoft’s market cap grew from $300 billion in February 2014 to over a trillion at the close of 2019.

“When we exercise a growth mindset by being customer-obsessed, diverse and inclusive and act as One Microsoft, that’s when we live our mission…”

– SATYA NADELLA, HIT REFRESH, HARPER COLLINS, 2017

Then what?

To what in the extent do marketers and consumers perceive brands as capable of delivering “an exceptional customer experience”? There is a marked mismatch. The gap ranges from 8% to 26%.

Source: eMarketer, March 2019

Consider the table above. Marketers and consumers diverge markedly in their perception of the quality of customer experience. We have to keep asking ourselves why.

Measuring CSAT or CX is not enough. Executives need to know the top three or four actions to take. CX analytics must yield insights, helping to identify customer needs and to prioritize actions. Decision makers need segmented insights linked to CX and financial outcomes.

We must also look wider, to understand economic and social behaviors. We are living through expanding sales and service channels, and through globalization. We have intricate networks of partnerships, and multiple media interactions. It is complex, and getting more so. Our models must incorporate the key components of this broader context. Then action planning can involve multiple initiatives across regions / channels. We thus ensure faster response times and create adaptive and flexible business processes.

“Companies that fail to adapt their people, processes, and products to this multiple-journey, multiple-experience environment will be left behind.”

– THE NEW NORMAL – CUSTOMER EXPERIENCE FIRST, ADOBE DIGITAL MARKETING, JULY 2015

Businesses define KPIs. Increasingly, they will monitor progress via dashboards and will adjust implementation to meet targets. Tracked metrics can be both ‘micro’ and ‘macro’, corresponding to implementation at different levels of the organization. Metrics must be specific and measured at appropriate intervals.

Data is burgeoning. The challenge is (1) to obtain data that is reliable, consistent and relevant; and (2) to find the nuggets most important to success. Achieving such results will require transformation of organizational culture. There must be gradual but steady enhancement of technological and analytical competency.

This evolution towards analytics program maturity is illustrated in the chart below. 

Source: OCX Cognition. Expected growth in workload sophistication and scale as an analytics program matures.

Initially, we have basic reporting. Then analysis grows in sophistication; ‘why?’ questions are tackled. Modeling starts, and with it the ability to predict outcomes. Next we shift reporting to ‘now’: machine learning means that customer status is always up to date, at least in predictive form. Then customer-event triggers make the system instantly responsive, so that resources shift to where we need them, when we need them.

 

In short

We need to move CX analysis from just surveys to the entire data architecture – in particular to operational and behavioral customer data. For this we need particular data and technology, because:

  • Automation, our digital world, use of robots (in manufacturing, warehousing, etc.), and IOT devices yield new data that captures client and customer experiences; and
  • Modern technical advances in data storage, computational power, and analytics software make it possible to analyze the ‘Big Data’ now to hand.

Crunching all the data will provide a more holistic view of the customer experience, far beyond what surveys alone can offer. We have decided to call this new measurement system Spectrum NPS.

Businesses need to be on this trajectory. Will your organization transform its culture and processes?

ABOUT INSIGHTS FROM OCX COGNITION

OCX Cognition delivers the future of NPS. We ensure customer experience success by combining technology and data science with programmatic consulting. In our Insights section, we present a comprehensive and evolving collection of resources based on our research and expertise, collected for CX leaders committed to delivering business outcomes.

The Value of Repeat Responder Feedback is Immense

The Value of Repeat Responder Feedback is Immense

INSIGHTS
The Value of Repeat Responder Feedback is Immense
BRIEF

The Value of Repeat Responder
Feedback is Immense

Validate NPS trends by looking at the responses of individuals
who have also given feedback in the past.

Catherine is confused. She has just completed a deep study of the relationship between customer satisfaction and customer loyalty. There are surprises, some bad. Some customers whose NPS improved over the last 18 months did not renew their contracts. Yet other customers with sharp declines in scores remain loyal, even though the company did nothing special to retain them. What is going on?

Catherine leads customer experience for a large telecoms provider. It specializes in selling VoIP solutions to corporations. She has been with the company for two years and reports to the CEO. She now fears for her job. How has NPS lost predictive value? She has had years of successful financial forecasts with two other companies. Could they have been exceptions in some way?

 

Best practices

In this article we are looking at relationship research using survey-based NPS. Spectrum NPS will automate and greatly enhance NPS analytics. However, to use NPS you also need a feel for how it works at the basic data level.

To proceed with our story… You follow best practices in customer experience research. You have analyzed feedback from your relationship survey and you have a Net Promoter Score at brand level. You also have other satisfaction / agreement scores for the elements in your customer journey. From this data you find the top drivers of loyalty. You implement improvement activities to increase satisfaction and loyalty.

After a period of time, the next wave of the relationship survey comes in. The first thing everyone wants to know is “Has our score improved?” That bare question is easy to answer. But even if your score has improved, does the movement represent a true improvement in the experience you are delivering? If you have surveyed a representative sample of your customers in each wave, it is fair to assume that an increase in score is real.

Bear in mind that your customer base is changing. New customers in the honeymoon period are included in the sampling. Ones who have not renewed their contract have disappeared. Things change.

 

Comparing apples to apples

One really valuable analytic is repeat responder analysis. It applies in both B2B and B2C situations. Find customers / contacts who have responded in both periods. Their feedback will tell you at the individual client level whether you have improved (or not), and where. This data will provide a sub-plot to the headline story of your results.

 

Methodology

While it is possible to consider the actual scores given, it can be time-consuming to identify all the changes. Luckily, it is sufficient to look at movement by category, namely Promoters, Passives and Detractors. The procedure can differ somewhat depending on whether the data are B2B or B2C.

But even if your score has improved, does the movement represent a true improvement in the experience you are delivering?

Recommended approach for B2C, or where B2B responses by account are limited

From the list of respondents from two survey waves, identify contacts who have responded twice. Do this using email address or, ideally, customer ID (which is unlikely to change even if the email address has). For each wave, note whether they were a Promoter, Passive and Detractor, then calculate:

  • the number of Detractors in Wave 1 who are Detractors in Wave 2;
  • the number of Detractors in Wave 1 who are Passive in Wave 2;
  • and so on – see the example below.

The example table below shows NPS movement for these respondents in Wave 1 and Wave 2. The change is a truer trend of your NPS. So how did the change come about? Well, a change in the proportion of Detractors, Passives and Promoters (obviously!). What is interesting is what percent of respondents changed (or didn’t).

 

 

Wave 2 NPS Category

Detractor (n=141) 

Passive (n=392) 

Promoter (n=395) 

Wave 1 

NPS Category 

Detractor (n=145) 55%34%11%

Passive (n=407) 

13%60%27%

Promoter (n=376) 

2%26%72%

 Total number of repeat respondents = 928

 

 

Recommended approach for B2B

In B2B, the account rather than individuals is the ‘decision making’ or purchasing unit. The aggregate of the respondents in an account represents the likelihood of the account staying loyal, purchasing more, and so on.

In tracking repeat responders, the focus is only on determining whether the account is overall Promoter, Passive or Detractor. From the list of responses in Wave 1 and Wave 2, identify company accounts who have at least one respondent in both waves. (For a more robust analysis, you could select only those accounts with two or more responses, or even three or more).

In B2B, the account rather than individuals is the ‘decision making’ or purchasing unit.

Use this simple rule of thumb:

  • if there are more Promoters than Detractors in the account, it’s Promoter;
  • if there are more Detractors, it’s Detractor;
  • if there is the same number of Promoters and Detractors, it’s Passive.

As above, calculate the number of Detractor accounts in Wave 1 and what category they are in Wave 2. Do the same for Passive and Promoter accounts.

 

Displaying the results

One way of visualizing the results is an Alluvial Diagram. It depicts the flow, or change of proportions, from one wave to the next. The thickness of the flow represents the volume (in this case the percentages). The chart below shows the data from the example above.

We can see that 72% of Promoters in Wave 1 were still Promoters in Wave 2. They are the most consistent group, so investment in creating Promoters pays off. It also shows that the danger with Passives (who are often ignored); they can become Promoters but also Detractors.

The data behind this chart (i.e., the actual contact names) can be used to contact the nine Promoters as well as the 52 Passives who all became Detractors. However, remember not to mention the NPS categories when speaking with clients!

 

Is it the same picture across all segments?

If you have sufficient repeat respondent data, segment your analysis to uncover variations in the trends. For example:

  • Is there a lifecycle impact? Do customers stay loyal in their first one to two years of tenure and then decline?
  • Are larger customer respondents more likely to remain the same than smaller?
  • Does purchase of a specific product or service impact the likelihood to increase / decrease loyalty?
  • We know there is a cultural / geographical impact on NPS. Is there a different pattern of repeat respondents in various geographies and do you have enough responses to determine that accurately?

What to do?

Where customer loyalty is declining, it is important to understand the root cause. It could be the result of one more poor service experiences, or it could be more fundamental. Ask yourself whether your product or service offering is still appropriate. Ask yourself whether an element of your service for these customers has reduced.

Similarly, looking at the customers who have become more loyal, identify whether you took any specific actions. Or did they change their relationship with you? Identify accounts or segments of customers who have the potential to increase their loyalty and focus efforts on them. If, as in the example above, Promoters are the most likely to remain Promoters, then your efforts will be rewarded.

Depending on the change (or not) in the NPS categories, the following table suggests a course of action.

 

 

Wave 1Wave 2Action
PromoterPromoterThese customers are satisfied with your services, especially those which are driving loyalty. They have indicated they would recommend you (twice). Develop opportunities for them  reference programs, testimonials, as well as contribution to product / service design and pilot programs. 
PromoterPassiveThese customers are not feeling so strongly about their willingness to recommend. It’s important to understand what has changed (is it you or is it them?). Communicating with them will uncover the causes and help you convert them back to Promoters. 
PromoterDetractorThis situation is a concern, especially if it includes high value customers. Contact them immediately in order to understand the cause for this severe decline in loyalty.  
PassivePassiveThese customers have been converted to Promoters. You should understand why (because you want to continue taking similar actions with other customers in future). Don’t forget to thank them, and make them feel special by revealing the opportunities for Promoters (as above). 
PassivePromoterCustomers are not feeling delighted by the products and services you provide. While not actively speaking against you, they are looking around at alternative suppliers. If they are particularly valuable, or in a target segment, you should consider the risk of them leaving you. 
PassiveDetractor These customers are now likely to advise their friends or colleagues not to do business with you. Because they were passive, they will have a reason for their decline which they will undoubtedly share.  
DetractorPromoterThis may be a small minority of customers, but it will be important to understand how they have become Promoters. In a B2B environment, the account manager may already be familiar with changes in contacts, or a particular event which triggered this. If not, contact the customer! 
DetractorPassiveThe conversion of Detractor customers to Passive is the first hurdle to improving your overall NPS score.  
DetractorDetractorThe danger with customers who remain Detractors from one wave to another is that they may become more severe. They may advocate more frequently against purchasing your products or services. It is important to keep in contact with them. Do this not only to address the issues causing detraction, but also communicate what you have done. 

 

With B2B customers, do not just look at the change in overall loyalty. Make sure that the account manager is tracking individual respondents, to understand the mood of the decision makers and influencers.

 

Understanding your ideal customer

Tracked over several years, we can use this analysis to understand customers who never increase their loyalty. Some people / customers will always be Detractors and you may determine a particular strategy for managing them (or maybe just let them go).

For Promoter customers who stay Promoters, do they share any common characteristics? Can you profile the ideal Promoter and target your acquisition efforts towards increasing your Promoter base? Acquiring customers is expensive, so targeting people and companies you know are likely to be or to quickly become Promoters is a wise investment.

Identifying the ideal Promoter and the likely Detractor has an added bonus when looking for new customers. For example, expending resources on customers who are likely to be Detractors could be very wasteful. Being able to sniff them out before signing them up is a potential big win.

After establishing the best type of customer, you should also be able to construct the best packages of product and service to offer them. Think too of creating the best package for Passive customers, to encourage them to become Promoters.

All this will be quite easy to explain to her boss. The financial predictions based on repeat responders alone look pretty positive. Quite a relief.

Back to Catherine

So, what had happened to our Catherine? Once she separated the repeat responders from the first-time responders, there were almost no surprises left.

There were situations where relatively positive answers from new respondents masked deteriorating views of people who had answered in Wave 1. And the new people turned out to have less influence on purchase decisions than those who had answered several times over the years. Fortunately, the same applied in the opposite direction, with positive repeat responders ensuring contracts got renewed.

All this will be quite easy to explain to her boss. The financial predictions based on repeat responders alone look pretty positive. Quite a relief.

Source: OCX Cognition.

ABOUT INSIGHTS FROM OCX COGNITION

OCX Cognition delivers the future of NPS. We ensure customer experience success by combining technology and data science with programmatic consulting. In our Insights section, we present a comprehensive and evolving collection of resources based on our research and expertise, collected for CX leaders committed to delivering business outcomes.

B2B CX Measurement Differs Radically from B2C

B2B CX Measurement Differs Radically from B2C

INSIGHTS
B2B CX Measurement Differs Radically from B2C
BRIEF

B2B Customer Experience
measurement differs radically from B2C

Most of what you know about B2C CX must be done differently for B2B.

We are all familiar with the survey requests we consumers receive by email, and with pop-up feedback requests on websites. Despite low response rates, these are effective feedback mechanisms for two simple reasons:

 

  • First, we, the people who receive the feedback requests, are indeed the people who make the purchase decisions.
  • Second, we are all pretty much equal. My feedback to Amazon as an end customer has about the same value to Amazon as the feedback from any other end customer.

Five reasons B2B CX measurement is different

Things are not so simple in B2B. Our temptation to just push out the surveys to whatever contact list we happen to have is wrong, or at least sub-optimal. Here are five reasons:

1

End User

The people who use your products or services are often not the same as the people who buy them. Think about a company that provides restaurant services to its clients. The end-users – the people who eat the food in Company X – have no role at all in the supplier selection. The CEO of X and the leadership team may have no role either. Sole responsibility may belong to a procurement manager, Sam. Sam’s only objective is that any new contract should cost less than the previous one. You need feedback from Sam, or your research will be worthless.

2

Who for What:

If yours is a large company with many products and services and your client is also a large company, things change again. Each of your products or services may be sold to a different part of your client company. Asking for feedback about individual products and services may be relatively easy. But whom do you ask for overall brand-level feedback?

3

Moving target:

People at your client company may change jobs. Your contact and feedback request list must be current. And beware: if you send your request by email to someone who has changed jobs they may still respond, but based on obsolete knowledge.

4

Captives:

It may be hard for your clients to change vendors – both administratively and emotionally. Your customers want you to be successful. Those who chose your company have invested some personal reputation in the decision, particularly if the purchase is substantial. They may hesitate to give you honest negative feedback. They may be particularly reluctant if the feedback will become known to others in their organization.

5

Size Rules:

The various issues are intertwined. But one that affects all the others is the relative size of customers. You must handle your largest customers differently. They know their importance and they expect to be treated as special.

You must handle your largest customers differently. They know their importance and they expect to be treated as special.

Consider the way you ask people for feedback. Should you really just send the same email to someone who spent a million dollars with you as to a person who spent $10k? (No!) Should you send a feedback request to just a single person in both companies? (Again, No!)

 

Differentiate your approach

Your research and improvement process is a strategic weapon that can give you a competitive edge. For your most important customers, make sure your process is more effective than what your competitors use.

The metrics must be reliable. You must keep track of how the same people respond in successive research waves. Your process must produce credible, customer-specific improvement suggestions.

Consider interviewing your top customers face-to-face. If you have a corporate executive sponsor program, think about asking the executive sponsors to do the interviews. Use the executive sponsors to agree improvement actions for that specific customer.

Our temptation to just push out the surveys to whatever contact list we happen to have is wrong, or at least sub-optimal

Use your own operational metrics

Too many CX initiatives rely only on survey data. Day-to-day operational data is usually at least as important. If you are a product company, how has your delivery performance been trending? Have the customers had a lot of warranty issues? If you are a service company, how has your performance measured up to what is specified in the service level agreement?

Weight the data for reporting

If your company has some very large customers and many much smaller ones, weight the data by revenue. But note that a statistically complex approach is difficult to explain. We suggest you present three numbers, or ideally three trends:

  • The overall number or trend for all customers, without any weighting.
  • The ‘Top 20’ number or trend. (This can also be ‘Top 50’ or whatever is the number of big customers that you feel deserve a special approach.)
  • The number or trend for ‘the rest’, meaning everyone except the Top 20.

One last subtlety to consider

You may want to consider whether your Top 20 should be the customers you would like to have as Top 20, rather than your current list. If so, estimate how much the various customers spend on your type of products and services. Note whether they spend that money with you or with your competitors. You could pick, for example, five customers who spend very large sums with your main competitor. Use your CX measurement and improvement process to capture them.

Always remember: your measurement and improvement process can be a competitive weapon!

ABOUT INSIGHTS FROM OCX COGNITION

OCX Cognition delivers the future of NPS. We ensure customer experience success by combining technology and data science with programmatic consulting. In our Insights section, we present a comprehensive and evolving collection of resources based on our research and expertise, collected for CX leaders committed to delivering business outcomes.