Stochastics and StatisticsBehavior-aware user response modeling in social media: Learning from diverse heterogeneous data
Introduction
Mass marketing and direct marketing are two commonly used approaches for product (service) advertising and promotional activities (Bose & Chen, 2009). For direct marketing, a marketing message is delivered to target customers without an intermediary person or indirect media involved (Bose & Chen, 2009). Customer response modeling aims at identifying the target customers who will respond to a specific marketing campaign from the existing customer base (Cui, Wong and Zhang, 2010, Kang, Cho and MacLachlan, 2012). With more and more companies adopting direct marketing, customer response modeling has become one of the most effective direct marketing strategies to increase total revenue and decrease marketing cost (Cui, Wong and Lui, 2006, Kang, Cho and MacLachlan, 2012, Lee et al., 2010). Because the purpose is to identify customers as possible respondents and non-respondents to a specific marketing campaign (Bose and Chen, 2009, Lee et al., 2010), customer response modeling is a binary classification problem.
For customer response modeling, external and behavioral data are usually used (Bose & Chen, 2009). Customer demographic, geographic and lifestyle data are often obtained from external data vendors (Baecke & Van den Poel, 2011), and thus are called external data. Customer behavioral data including transaction records, feedbacks to marketers, customer reviews and Web browsing records are considered to be the most important data in customer response modeling (Bose & Chen, 2009). Many supervised and semi-supervised machine learning techniques have been proposed for the customer response modeling problem (Lessmann & Voß, 2008). These techniques include artificial neural networks (ANN) (Crone, Lessmann and Stahlbock, 2006, Kim et al., 2005), decision trees (Crone et al., 2006), Bayesian networks (Baesens et al., 2002, Cui, Wong and Lui, 2006), logistic regression (Kang et al., 2012), bagging (Ha, Cho, & MacLachlan, 2005), support vector machines (SVM) (Crone, Lessmann and Stahlbock, 2006, Kang, Cho and MacLachlan, 2012, Lessmann and Voß, 2009) and transductive SVMs (Lee et al., 2010). Moreover, some other techniques including clustering (Kang et al., 2012), sampling (Crone, Lessmann and Stahlbock, 2006, Kang, Cho and MacLachlan, 2012), sequential pattern discovery (Chen, Hsu, & Hsu, 2011), feature selection (Cui et al., 2010) and other preprocessing methods (Crone et al., 2006) have been combined with classification techniques to refine the customer base and improve prediction performance.
In the age of Web 2.0, social media sites develop rapidly. Social media refers to a group of online applications allowing the creation and exchange of user-generated contents (Kaplan & Haenlein, 2010). The most popular types of social media include wikis, blogs, microblogs, social networks, video and photo sharing and online communities. They become popular communication tools due in part to the open access of the Internet, the popularity of mobile devices, the availability of the tools and the fast social interactions among users. Social media have increasingly become a major factor influencing the opinions, attitudes and the purchase behavior of customers (Mangold & Faulds, 2009).
User behavioral data generated and collected on social media sites include two categories, i.e., individual behavioral data and engagement behavioral data. Moreover, according to the ways of using the behavioral data in the customer response models, user behavioral data can be classified as longitudinal behavioral data and aggregated behavioral data. Fig. 1illustrates the different types of behavioral data. For traditional customer response modeling, the longitudinal individual behavioral variables derived from the transactional databases are usually transformed into the aggregated variables such as recency, frequency and monetary (RFM) variables which have been included in most direct marketing datasets and adopted in most response models (Baesens et al., 2002, Crone, Lessmann and Stahlbock, 2006, Cui, Wong and Zhang, 2010).
In comparison with individual behavior, customer engagement behavior, as an emerging concept, focuses on the customers’ behavioral manifestation beyond purchase such as electronic word-of-mouth, customer–customer interaction, recommendations, blogging and online reviews (van Doorn et al., 2010). In social media, customer engagement behavior has great effect on the individual purchase decisions (Cheung and Thadani, 2012, Dellarocas, 2003, van Doorn et al., 2010). For example, Dell gained high income by posting offers to its followers on Twitter (Li & Li, 2013). A survey showed that 91 percent of respondents said that they consulted online reviews before purchasing, and 46 percent of respondents believed that the online reviews influenced their purchase decisions (Cheung & Thadani, 2012). Therefore, incorporating engagement behavioral data into customer analytical models is increasingly recognized as a new direction of customer relationship management and direct marketing (Bijmolt et al., 2010).
The aggregated individual behavioral attributes are usually used as predictors in most customer response models. Few existing studies of customer response modeling pay attention to the longitudinal individual and engagement behavioral data which are widely available in the social media databases. In recent years, the analysis of engagement behavior has been used widely in the areas of recommendation and customer churn prediction. Some researchers combined the extended factorization model with other methods such as additive forest, logistic regression and scorecard model to predict the top-N items the customer was most likely to follow using the aggregated customer–customer interaction data (Chen et al., 2012, Chen et al., 2012, Zhao, 2012). The information of individual customers and a group of customers which have similar characteristics was used in a novel customer profile model for product recommendation (Park & Chang, 2009). For customer relationship management of the telecommunication industry, the customer–customer interaction data have been recognized as important complements to traditional behavioral data. The aggregated engagement behavioral attributes were combined with traditional attributes to predict customer churn (Zhang, Zhu, Xu, & Wan, 2012).
Some researchers recognized that customer purchase behavior varies over time and the use of the longitudinal individual behavioral data can improve prediction performance (Chen, Fan and Sun, 2012, Liu, Lai and Lee, 2009). Sequential pattern analysis was combined with collaborative filtering for temporal purchase behavioral data to improve recommendation performance (Cho, Cho and Kim, 2005, Choi et al., 2012, Huang and Huang, 2009, Liu, Lai and Lee, 2009, Min and Han, 2005). Prinzie and Van den Poel (2006, Prinzie and Van den Poel, 2007, Prinzie and Van den Poel, 2011) incorporated customer purchase sequence into dynamic Bayesian networks and Markov models to predict the next product for a customer to buy. Ballings and Van den Poel (2012) studied the problem of how long the customer historical data should be for customer churn prediction. They suggested that selecting a good length of historical data can decrease computational burden.
For social media, the term Item may represent a specific user, organization, product (service) or event. Examples of events include the appearance of a new term or keyword, the announcement of a new product (service) or activity, or a new price of an existing product (service). The rich behavioral data generated on social media sites can be used for managers to predict user responses to an Item, make marketing policies and allocate marketing resources to influence customer behavior (Power & Phillips-Wren, 2011). For social media, customer response modeling is also called user response modeling, and the two terms are used interchangeably. In this study, customer response modeling taking into consideration of user behavioral, e.g., longitudinal individual and engagement behavioral, data is called behavior-aware user response modeling. However, the large, diverse and heterogeneous data generated on social media sites bring great challenges on behavior-aware user response modeling (Bijmolt et al., 2010, Cao, Ou and Yu, 2012, Chau and Xu, 2012).
How to deal with diverse heterogeneous data is a challenge. A variety of methods can be used for customer response modeling using external and aggregated individual behavioral data. However, to the best of our knowledge, this study is the first attempt of combining the individual behavioral and the engagement behavioral data, as well as the longitudinal and the external data for user response modeling in social media.
How to deal with large amount of data is another challenge. Social media sites produce large amount of user data. For example, the daily volume of posts mentioning some well-known brands or products such as Google, Microsoft, Sony, iPhone and iPad in Twitter is in the millions (Li & Li, 2013). It is necessary to use marketing intelligence methods to automatically analyze the massive amount of data. The analysis of the massive amount of data requires efficient preprocessing of the data and excellent scalability of the customer response models.
In this study, a hierarchical ensemble learning framework is developed for behavior-aware user response modeling in social media. In the framework, a general-purpose data preprocessing strategy is proposed to transform the large-scale and multi-relational user datasets derived from social media sites into high-order tensors and to extract attributes as input of the models. An improved hierarchical multiple kernel SVM (H-MK-SVM), as an extension of the SVM and the multiple kernel SVM (MK-SVM), is developed to model diverse heterogeneous data including external, tag and keyword, individual behavioral and engagement behavioral data. Because of the multi-relations of the individual behavioral and engagement behavioral data, one advantage of the improved H-MK-SVM is its ability in adaptively selecting associated attributes. Another advantage of this method is its ability in integrating the diverse heterogeneous social media data into a unified ensemble classifier to improve the prediction performance. Furthermore, the subagging strategy (Paleologo, Elisseeff, & Antonini, 2010) is adopted to deal with large imbalanced datasets and ensemble methods are used to combine the results of subagging.
This paper is organized as follows. The next section presents the hierarchical ensemble learning framework for behavior-aware user response modeling in social media. Section 3 describes the database used in the study and presents the data preprocessing strategy. The model formulation of the improved H-MK-SVM is presented in Section 4. The computational results are reported in Section 5. Conclusions and directions for future research are given in Section 6.
Section snippets
The hierarchical ensemble learning framework
In this section, diverse heterogeneous data used for user response modeling in social media are discussed. A hierarchical ensemble learning framework is then proposed for user response modeling in social media using the diverse heterogeneous data.
The data
In this section, the database used for the computational experiments is introduced first. The data transformation and feature extraction strategies for heterogeneous high-dimensional and multi-relational datasets are then described in detail using this database. The subagging strategy is then discussed.
The model
An improved H-MK-SVM, based on the work of Chen, Fan, et al. (2012), is developed in the hierarchical ensemble learning framework. The H-MK-SVM is an extension of the SVM and the MK-SVM. The SVM is one of the most popular and effective machine learning techniques and usually has excellent classification performance in practical applications (Vapnik, 1998). The MK-SVM, as an important extension of the SVM, can integrate heterogeneous data and adaptively select the best combinations of multiple
Computational experiments
Computational resultsare reported in this section. Matlab 7.4 was used to conduct the computational experiments. The desktop computer used for the computation has an Intel Core i7 processor with a 3.40 gigahertz clock speed and has 16 gigabytes of RAM. Eight competitive methods including the SVM, feed-forward ANN (FFANN), radial basis function neural network (RBFNN), decision tree (DT), random forest (RF), Adaboost and the two ensemble methods, i.e., MV and AV, were also used in the experiments
Conclusions
In this study, a hierarchical ensemble learning framework is developed for behavior-aware user response modeling in social media using diverse heterogeneous data. In the framework, a general-purpose data transformation and feature extraction strategy is proposed and an improved H-MK-SVM is developed. In comparison with the work of Chen, Fan, et al. (2012) on customer churn prediction, the major contributions of this study are (1) for the original data from social media, a data transformation
Acknowledgments
The authors greatly appreciate the three anonymous reviewers for their constructive suggestions. This work was partially supported by the National Natural Science Foundation of China (Project Nos. 71101023, 71471035 and 71271051) and the Fundamental Research Funds for the Central Universities, NEU, China (Project No. N120406001).
References (49)
- et al.
Bayesian neural network learning for repeat purchase modelling in direct marketing
European Journal of Operational Research
(2002) - et al.
Customer event history for churn prediction: How long is long enough?
Expert Systems with Applications
(2012) - et al.
Quantitative models for direct marketing: A review from systems perspective
European Journal of Operational Research
(2009) - et al.
Customer-adapted coupon targeting using feature selection
Expert Systems with Applications
(2004) - et al.
Handling class imbalance in customer churn prediction
Expert Systems with Applications
(2009) - et al.
Optimal selection of potential customer range through the union sequential pattern by using a response model
Expert Systems with Applications
(2011) - et al.
A hierarchical multiple kernel support vector machine for customer churn prediction using longitudinal behavioral data
European Journal of Operational Research
(2012) - et al.
The impact of electronic word-of-mouth communication: A literature analysis and integrative model
Decision Support Systems
(2012) - et al.
Mining changes in customer buying behavior for collaborative recommendations
Expert Systems with Applications
(2005) - et al.
A hybrid online-product recommendation system: Combining implicit rating-based collaborative filtering and sequential pattern analysis
Electronic Commerce Research and Applications
(2012)
The impact of preprocessing on data mining: An evaluation of classifier sensitivity in direct marketing
European Journal of Operational Research
Bayesian variable selection for binary response models and direct marketing forecasting
Expert Systems with Applications
Response models based on bagging neural networks
Journal of Interactive Marketing
Handling sequential pattern decay: Developing a two-stage collaborative recommender system
Electronic Commerce Research and Applications
Improved response modeling based on clustering, under-sampling, and ensemble
Expert Systems with Applications
Users of the world, unite! the challenges and opportunities of social media
Business Horizons
Semi-supervised response modeling
Journal of Interactive Marketing
A reference model for customer-centric data mining with support vector machines
European Journal of Operational Research
Deriving market intelligence from Microblogs
Decision Support Systems
A hybrid of sequential rules and collaborative filtering for product recommendation
Information Sciences
Social media: The new hybrid element of the promotion mix
Business Horizons
Detection of the customer time-variant pattern for improving recommender systems
Expert Systems with Applications
Subagging for credit scoring models
European Journal of Operational Research
Individual and group behavior-based customer profile model for personalized product recommendation
Expert Systems with Applications
Cited by (22)
Machine learning algorithms for social media analysis: A survey
2021, Computer Science ReviewCitation Excerpt :By taking the help of network theory, we implement the network analysis, which plays a significant role in refining, identification of animal movement ecology, and behavior. In [95], the authors described a behavior-aware user response system. They use a hierarchical ensemble learning framework for heterogeneous SM data.
Differences in online reviews caused by distribution channels
2021, Tourism ManagementCitation Excerpt :Bilgihan, Barreda, Okumus, and Nusair (2016) suggested that social network services generate collective knowledge and are considered as information-gathering sources among tourists when they make travel-related decisions. Chen, Fan, and Sun (2015) showed that user-generated content on social media could play a role in growing product awareness among prospective customers. This has the potential to directly contribute to an increase in sales.
On-line classification of coal combustion quality using nonlinear SVM for improved neural network NOx emission rate prediction
2020, Computers and Chemical EngineeringWhen blockchain meets social-media: Will the result benefit social media analytics for supply chain operations management?
2020, Transportation Research Part E: Logistics and Transportation ReviewDeveloping a smart operational research with hybrid practice theories
2019, European Journal of Operational ResearchCitation Excerpt :This is of particular relevance to OR as marketing is a fast-growing area at the interface of OR with other disciplines (Jin, Wu, & Hu, 2017; Karray & Martín-Herrán, 2018; Nalca et al., 2018; Pnevmatikos, Vardar, & Zaccour, 2018; Torres & Bijmolt, 2009; Zhang, Zhang, Cheng, & Hua, 2018). Relatedly, Carah (2017) traces how culturally embedded, and participatory forms of branding have become integrated with the predictive and analytic capacities of social media, which is of relevance to OR (Chen, Fan, & Sun, 2015). However, with the emergence of data-driven technologies, Carah argues that the logic of participation has shifted (cf. Arvidsson & Peitersen, 2013 cited in Carah, 2017).
Individual-level social influence identification in social media: A learning-simulation coordinated method
2019, European Journal of Operational Research