Churn and Net Promoter Score forecasting for business decision-making through a new stepwise regression methodology

https://doi.org/10.1016/j.knosys.2020.105762Get rights and content

Highlights

  • Novel business-based classification methodology.

  • Winner of a competitive international analytic challenge.

  • Good applicability in other contexts.

Abstract

Companies typically have to make relevant decisions regarding their clients’ fidelity and retention on the basis of analytical models developed to predict both their churn probability and Net Promoter Score (NPS). Although the predictive capability of these models is important, interpretability is a crucial factor to look for as well, because the decisions to be made from their results have to be properly justified. In this paper, a novel methodology to develop analytical models balancing predictive performance and interpretability is proposed, with the aim of enabling a better decision-making. It proceeds by fitting logistic regression models through a modified stepwise variable selection procedure, which automatically selects input variables while keeping their business logic, previously validated by an expert. In synergy with this procedure, a new method for transforming independent variables in order to better deal with ordinal targets and avoiding some logistic regression issues with outliers and missing data is also proposed. The combination of these two proposals with some competitive machine-learning methods earned the leading position in the NPS forecasting task of an international university talent challenge posed by a well-known global bank. The application of the proposed methodology and the results it obtained at this challenge are described as a case-study.

Introduction

Client churn, the possibility that current clients will terminate their relationship with a company, is one of the most serious issues faced by any company, regardless of industry. Client churn is usually due either to episodes of mismanagement, which cause client relationships to deteriorate, or to actions by competing companies, such as offering clients more appealing products or services. To address this issue, many analytical models for predicting client churn have been developed.

In these models, the target variable to be predicted is usually the client churn itself, that is, a binary variable that takes the value of 1 when the customer has left the company, and 0 otherwise. However, another tactic is to predict the Net Promoter Score (NPS, see [1]) metric. This indicator is a commercial brand registered by Frederick Reichheld, Bain & Company, and Satmetrix, and was introduced by Reichheld himself in 2003 in his article “The one number you need to grow” [2]. The NPS of a company is primarily obtained through client surveys, which typically ask for a numeric valuation, from 0 to 10, associated to the question “How likely are you to recommend the company‘s products/services to a relative or a friend?”.

In order to develop appropriate marketing campaigns, the NPS metric is typically discretized into three categories in such a way that NPS values between 0 and 6 identify “detractor” clients, values of 7 or 8 are associated with “neutral” clients, and values of 9 or 10 are considered “promoters”. The prediction of this discretized NPS, therefore, sets a 3-class classification problem. As clients must be surveyed to determine their NPS, it is generally infeasible to obtain an NPS value for all clients for several reasons (e.g. cost, low penetration index, answering biases, etc.). Thus, employing analytical models to predict (discretized) NPS is the only practical solution to obtain NPS values for all clients.

It is also important to note that, in a marketing context, the interpretability of the analytical models to be applied is a highly relevant matter [3]. The dependency relationships between the indicator to be predicted (such as churn or NPS) and the explanatory variables must be easy to understand. This is because decisions have to be made taking these relationships into account. For example, in a banking company, if a client churn prediction model indicates that clients who pay commissions have a higher churn probability (or are more prone to be detractors), the company may then decide to eliminate this payment, especially for its most valuable clients.

Within this context, the main contributions of this work are the following:

  • -

    First, an analytical modelling methodology called business stepwise is proposed, which is a variant of the classical stepwise variable selection method widely used in fitting regression models. The proposed variant performs a variable selection process that considers not only the statistical significance of the variables to be included in the model, but also their business logic, which has been previously validated by an expert, as well as the correct specification of that logic in the model.

  • -

    Second, a new method for transforming input variables when predicting a multiclass ordinal target (like the discretized NPS indicator) is proposed, which simplifies the proposed variable selection method.

These methods were applied to address a problem set up by a well-known global banking company in an international university talent challenge [4] in which several university teams competed. The challenge was structured in two tasks, respectively aimed at the development of an original analytical methodology to predict client churn and the discretized NPS indicator. The design of an easily interpretable methodology was a critical factor stipulated by the banking company in order to evaluate the submissions to the challenge. This motivated the development of the methodology proposed in this study. The fact that the accuracy of predictions was obviously another relevant factor in the evaluation criteria, led us to combine the predictions provided by our method with those obtained using other machine learning techniques. The method’s originality, together with the interpretability and accuracy of its results, earned it the leading position in the discretized NPS prediction task of the challenge.

The paper is organized as follows. Section 2 gives a brief review of the most extended supervised classification methods. In Section 3, we describe the proposed business stepwise variable selection methodology in detail along with our proposed method for transforming the input variables when predicting a multiclass ordinal target. In Section 4, the application of the proposed methodology to the NPS forecasting task of the mentioned university talent challenge is exposed as a case-study. Finally, Section 5 is devoted to present the most relevant conclusions of our work.

Section snippets

Supervised classification: Interpretable and predictive methods

Several strategies can be followed to fit a supervised classification model in which the target or dependent variable has a categorical nature. For instance, in the case of a multiclass ordinal target (e.g. the discretized NPS), classification models can be fit either by taking into account or omitting this ordinal nature. Similarly, a multiclass classification problem can be decomposed into a family of binary classification problems, which can be then addressed separately by means of binary

A business-based stepwise regression methodology

Any supervised modelling methodology oriented towards decision-making has to depart from a descriptive analysis that validates the available data. In this validation process, the analyst has to check if the distribution of the explanatory variables shows the expected behaviour, as well as if their relationship with the target presents the expected business logic. In this section, we propose a regression fitting methodology in which only explanatory variables reflecting the adequate, validated

A case study: Net promoter score forecasting

The following is the application of the proposed methodology to explain and predict a bank clients’ NPS (or satisfaction level), task two of the university talent challenge introduced in Section 1. The development of an analytical model for the purpose of predicting NPS has to be carried out in several stages:

  • 1.

    First, the company draws samples of clients to which the satisfaction survey is delivered on a given temporal basis, for instance quarterly.

  • 2.

    In a second stage, the company determines a list

Conclusions

In this work, a novel methodology for variable selection focused on model interpretability has been proposed in the context of logistic regression with WOE variables. To this aim, a new definition of WOE variables for ordinal targets has been introduced, which allows avoiding some of the issues usually associated to the discretization of inputs through binary dummy variables, particularly simplifying any subsequent variable selection process. Besides, the specific definition of these WOE

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work has been partially supported by the Government of Spain (grants TIN2015-66471-P and PGC2018-096509-B-I00), the Government of Madrid, Spain (grant S2013/ICE-2845), and Complutense University of Madrid, Spain (research group 910149 and project PR26/16-21B-3).

References (23)

  • RudinC.

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

    Nat. Mach. Intell.

    (2019)
  • Cited by (6)

    • HECON: Weight assessment of the product loyalty criteria considering the customer decision's halo effect using the convolutional neural networks

      2023, Information Sciences
      Citation Excerpt :

      The customers answer a question by using the 0–10 scale (a score of 0 is not at all likely, and a score of 10 is extremely likely). The respondents were categorized into three groups [24]: detractors (score 0–6), passives (score 7–8), and promoters (score 9–10) [2,25]. As mentioned earlier, customer loyalty is multi-dimensional and depends on multiple criteria.

    • Predictive machine learning for prescriptive applications: A coupled training–validating approach

      2022, Knowledge-Based Systems
      Citation Excerpt :

      Another engineering application of this scheme is provided by Xu et al. in [23] where an ML-based wear fault diagnostic model is used to predict fault modes in marine diesel engines, which then enables taking quick actions to avoid severe accidents in ships. An example of business application is given in [24], where the authors introduced a new step-wise regression methodology to better manage a company’s customer churn and improve decision making in marketing to help customer retention. Note though that in most of such cases the prediction model is trained without regard for its eventual use in decision making.

    • Swarm intelligence goal-oriented approach to data-driven innovation in customer churn management

      2021, International Journal of Information Management
      Citation Excerpt :

      In addition, Kim et al. (2013) claim that most prediction models do not consider additional costs of churn management programs (see also Lejeune, 2001). The need to prepare a next step ahead towards decision-making is also discussed in Vélez et al. (2020). The conducted literature studies allow stating that there are still research gaps and open issues, including:

    View full text