NPS BASICS AND BEYOND
Great NPS Surveys, Poor Data: What Went Wrong?
Data Quality Standards

Great NPS Surveys, Poor Data: 
What Went Wrong?

key takeaway

A great NPS survey design is of little value if feedback collection is not properly executed. Establish protocols for data quality and modes for collection to ensure your organization will have the necessary level of confidence to direct decision making.

Best Practice

The time to being thinking about NPS survey data quality is before your surveys are live. Here is our checklist for monitoring data quality.

  • NPS Validity
    The validity of your survey-based Net Promoter Scores is based on an accurate implementation of the recommend question and correct calculation of the score. Likelihood to Recommend needs to be the first question presented; the scale needs to be an 11-point, 0-10 scale; 0 is on the left to avoid a positive bias. Correct NPS calculation is as follows: % of Promoters minus % of Detractors; presuming a category for non-respondents in order to include them in the calculation should never be done and invalidates your score.
  • Timeliness of Data Capture
    Data should be measured at the appropriate point in the customer journey, which for relationship data collection means the respondent has actually experienced the full journey. Additionally, make sure no major inaccuracies or biases are introduced due to timing choices, e.g. sending a relationship survey right before a planned price increase in the hopes of receiving a better score.
  • Comparable Data Sets
    Period to period comparisons should be based on substantially comparable data sets as much as possible.
  • Representative Respondents
    Respondent profiles and quantities need to accurately reflect the underlying business and likely linkage to overall financial performance. Response needs to be of sufficient level, though not necessarily statistically significant, to create confidence in scores. While statistical significance is not required, a reasonable threshold for participation needs to be set by segment and region.

When it comes to determining the right mode for feedback collection, it’s about striking the right balance across several factors such as cost, potential for bias, type of feedback being requested and audience preference. Some modes such as phone interviews are costly, susceptible to bias and inefficient for most audiences. Email on the other hand is inexpensive and without bias, but not appropriate for in the moment transactional feedback or relationship feedback for certain audiences. Digital mechanisms are recommended, and we expect the trend towards the use of mobile apps, kiosks and tablets to continue due to low cost, lack of bias and ease for respondents.

Ideally one mode which enables ease of outreach on your end and ease of response by your customers is feasible. Should you find a need to employ multiple modes to achieve a representative sample, it is critical that the survey questions and scales are consistent regardless of mode.

The question then becomes whether or not to weight results by mode. For business customers, where volume is generally low, we do not recommend weighting by mode for simplicity’s sake. However, in the case of consumers, where volumes are high, we recommend weighting by mode to normalize data within segments or regions if there is a material mix of modes. Note that this recommendation assumes adequate sample size with no other difference in the sample composition, and a statistically significant difference in scores.

go beyond basic nps survey execution

Customer survey feedback has standalone value, of course, but for visionary NPS programs, the deeper value comes from its utility for calibrating an AI-driven understanding of the customer experience. Treat your feedback execution as a critical element for building your complete data set, but don’t make the mistake of stopping once you have data from direct customer feedback. To get there, download our guide, take our comprehensive training course, or read about the future of NPS.

REPORT

The Complete Guide to NPS Basics and Beyond

TRAINING

Leading an Outcome- Oriented
CX Program

REPORT

The New NPS
Manifesto

REPORT

The Complete Guide to NPS Basics and Beyond

TRAINING

Leading an Outcome- Oriented
CX Program

REPORT

The New NPS Manifesto

ABOUT OCX COGNITION

OCX Cognition delivers the future of NPS. We ensure customer experience success by combining technology and data science with programmatic consulting. In our Insights section, we present a comprehensive and evolving collection of resources based on our research and expertise, collected for CX leaders committed to delivering business outcomes.