NPS Score root-causing: Single-select or multi-select questions

Augmenting open text questions with single-select and multi-select questions in NPS surveys

NPS Score root-causing: Single-select or multi-select questions

Augmenting open text questions with single-select and multi-select questions in NPS surveys

NPS Score root-causing: Single-select or multi-select questions
Home > Insights > NPS Score root-causing: Single-select or multi-select questions

Introduction

NET PROMOTER SCORE (NPS) SURVEYS are a cornerstone of customer experience (CX) measurement, providing insights into customer loyalty and satisfaction. While the core NPS question (“How likely are you to recommend our product/service?”) is straightforward, the follow-up questions designed to understand the reasons behind the score can significantly impact the quality of insights gathered. While the standard open-text comment box is the go-to approach and a must-have follow-up question to any NPS Score question, companies often augment this open text comment box with additional questions that aim to quantify selected drivers of the NPS Score.

Two common approaches for follow-up questions are single response and multi-response formats. Each has its own advantages and disadvantages, and the choice between them can depend on the type of NPS survey being conducted, the company’s CX maturity, and the specific insights sought.

Advantages and Disadvantages of Single SELECT Follow-Up Questions

Single select follow-up questions offer several advantages, particularly in terms of clarity and simplicity. By forcing respondents to choose the primary reason behind their NPS score, this approach provides a clear indication of the most critical issue, making it easier for organizations to prioritize actions and allocate resources effectively. The simplicity of analyzing data from single response questions is another benefit, as it allows for straightforward aggregation of results and easier identification of trends. However, this approach is not without its drawbacks. Limiting respondents to a single option may oversimplify complex experiences, leading to incomplete data that fails to capture the full scope of customer sentiment. Additionally, this method may miss secondary factors that also significantly influence customer satisfaction or dissatisfaction, potentially skewing the overall understanding of customer needs. There is also a risk of bias, as respondents might select the first or most obvious option, which could lead to data that does not accurately reflect their true experiences.

Inherent advantages and disadvantages of single-select vs multi-select questions in the context of NPS Score follow up questions

Advantages and Disadvantages of Multi-SELECT Follow-Up Questions

In contrast, multi-select follow-up questions allow for a more nuanced understanding of customer experiences by capturing multiple factors that contribute to their NPS score. This approach is particularly valuable in contexts where customer experiences are influenced by a variety of factors, as it provides a more comprehensive view of what drives satisfaction or dissatisfaction. By allowing respondents to select multiple factors, companies can gain insights into the different aspects of the customer journey that impact overall satisfaction. However, multi-select questions also come with their own set of challenges. Collecting data on multiple factors can lead to an overwhelming amount of information, making it difficult to identify the most critical issues. This complexity in analysis requires more advanced analytical skills and tools, which can be a challenge for companies with lower CX maturity. Moreover, with responses spread across multiple factors, it can be challenging to prioritize actions and focus on the most impactful issues, potentially diluting the effectiveness of CX initiatives.

Forced Ranking in Multi-SELECT Questions

Some researchers have proposed using forced ranking in multi-select questions to mitigate the lack of prioritization and focus inherent in this approach. Forced ranking requires respondents to rank multiple options in order of importance or relevance. While this method can help introduce some prioritization, it also introduces additional challenges. Forced ranking can be mentally taxing for respondents, leading to fatigue and less thoughtful responses. Additionally, it may compel respondents to prioritize factors that are equally important, resulting in data that doesn’t accurately reflect their true sentiments. There is also the potential for bias in responses, as the order of options can influence rankings, and the complexity of the task may deter respondents from completing the survey altogether, reducing overall response rates and data quality. Furthermore, analyzing and interpreting forced ranking data can be challenging, especially when trying to aggregate results across respondents, which adds another layer of complexity to the process.

Adding CSAT Ratings to Multi-SELECT Questions

In some cases, companies not only ask respondents to select multiple factors in multi-select questions but also require them to rate their satisfaction (CSAT) with each factor. This approach can offer additional benefits, particularly in providing granular insights into specific areas that drive customer satisfaction. By collecting CSAT ratings for multiple factors, companies can use regression analysis to model the relationship between these ratings and the overall NPS score. This allows for the identification of “derived importance,” helping to prioritize efforts on the factors that have the most significant impact on customer loyalty. However, this approach also presents additional risks. The added complexity of rating multiple factors can lead to survey fatigue, reducing the quality of the data collected. The increased effort required may also deter participation, leading to lower response rates. Respondents may struggle to accurately rate multiple factors, especially if they are asked to recall specific details about each one, resulting in cognitive overload and potentially less reliable data. Analyzing CSAT ratings across multiple factors introduces further complexity, making it more challenging to derive clear and actionable insights. There is also the risk of overfitting in regression models, which can lead to insights that may not generalize well to broader contexts.

Suitability Based on CX Maturity and Survey Type

The decision to use single or multi-select follow-up questions in NPS surveys should be guided by several factors, including the type of NPS survey, the company’s CX maturity, and the specific insights sought. Single response questions offer simplicity and clarity, making them suitable for transactional NPS surveys and companies with lower CX maturity. In contrast, multi-select questions provide more comprehensive insights, making them ideal for relationship and relative NPS surveys, as well as companies with higher CX maturity that have the resources and expertise to manage more complex data analysis.

Take-away

There is no better option than to have an open-text comment box after every NPS Score question, assuming you have the robust text analytics tools to extract insights from this unstructured data.

However, for companies that want to augment this method with questions that allow them to quantify certain driver parameters, the choice is between single-select or multi-select driver questions.

For companies with high CX maturity, the benefits of adding CSAT ratings to multi-select questions and leveraging regression analysis to understand derived importance can outweigh the risks, provided they have the necessary tools, expertise, and processes in place. However, for companies with lower CX maturity, the added complexity and potential downsides of multi-select approaches may outweigh the benefits, making simpler survey designs more practical and effective in gathering actionable insights without overwhelming the organization. In summary, the choice between single and multi-select follow-up questions in NPS surveys depends on the organization’s ability to manage and act on the data collected. While multi-select questions with CSAT ratings can provide deeper insights, they require careful consideration of the company’s CX maturity, resources, and analytical capabilities to ensure the benefits are realized without introducing significant risks.

Access our related white paper: Unveiling Bias in Customer Survey Designs to explore comprehensive strategies for avoid bias in your customer satisfaciton surveys and improve the insights generated from such surveys.

Register or Login to continue reading