skip to main content
10.1145/3580305.3599759acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

A Data-Driven Decision Support Framework for Player Churn Analysis in Online Games

Published: 04 August 2023 Publication History

Abstract

Faced with saturated market and fierce competition of online games, it is of great value to analyze the causes of the player churn for improving the game product, maintaining the player retention. A large number of research efforts on churn analysis have been made into churn prediction, which can achieve a sound accuracy benefiting from the booming of AI technologies. However, game publishers are usually unable to apply high-accuracy prediction methods in practice for preventing or relieving the churn due to the lack of the specific decision support (e.g., why they leave and what to do next). In this study, we fully exploit the expertise in online games and propose a comprehensive data-driven decision support framework for addressing game player churn. We first define the churn analysis in online games from a commercial perspective and elaborate the core demands of game publishers for churn analysis. Then we employ and improve the cutting-edge eXplainable AI (XAI) methods to predict player churn and analyze the potential churn causes. The possible churn causes can finally guide game publishers to make specific decisions of revision or intervention in our designed procedure. We demonstrate the effectiveness and high practical value of the framework by conducting extensive experiments on a real-world large-scale online game, Justice PC. The whole decision support framework, bringing interesting and valuable insights, also receives quite positive reviews from the game product and operation teams. Notably, the whole pipeline is readily transplanted to other online systems for decision support to address similar issues.

Supplementary Material

MP4 File (adfp069-2min-promo.mp4)
Presentation video - short version

References

[1]
Jaehuyn Ahn. 2020. A Survey on Churn Analysis. ArXiv abs/2010.13119 (2020).
[2]
Özden Gür Ali and Umut Ar?türk. 2014. Dynamic churn prediction framework with more effective use of rare event data: The case of private banking. Expert Systems with Applications 41, 17 (2014), 7889--7903.
[3]
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Ben- netot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al . 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. In- formation fusion 58 (2020), 82--115.
[4]
Valerio Bonometti, Charles Ringer, Mark Hall, Alex R Wade, and Anders Drachen. 2019. Modelling early user-game interactions for joint estimation of survival time and churn probability. In 2019 IEEE Conference on Games (CoG). IEEE, 1--8.
[5]
Zhengping Che, Sanjay Purushotham, Robinder Khemani, and Yan Liu. 2016. In- terpretable deep models for ICU outcome prediction. In AMIA annual symposium proceedings, Vol. 2016. American Medical Informatics Association, 371.
[6]
Hugh Chen, Scott Lundberg, and Su-In Lee. 2021. Explaining models by propa- gating Shapley values of local components. In Explainable AI in Healthcare and Medicine. Springer, 261--270.
[7]
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM, 785--794.
[8]
Houtao Deng. 2019. Interpreting tree ensembles with intrees. International Journal of Data Science and Analytics 7, 4 (2019), 277--287.
[9]
Usama Fayyad and Keki Irani. 1993. Multi-Interval Discretization of Continuous- Valued Attributes for Classification Learning. In 13th International Joint Confer- ence on Artificial Intelligence, Vol. 2. 1022--1027.
[10]
Ruth C Fong and Andrea Vedaldi. 2017. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE International Conference on Computer Vision. 3429--3437.
[11]
Nicholas Frosst and Geoffrey Hinton. 2017. Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784 (2017).
[12]
David L García, Àngela Nebot, and Alfredo Vellido. 2017. Intelligent data analysis approaches to churn as a business problem: a survey. Knowledge and Information Systems 51, 3 (2017), 719--774.
[13]
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. 2018. Local rule-based explanations of black box decision systems. arXiv preprint arXiv:1805.10820 (2018).
[14]
Ruocheng Guo, Lu Cheng, Jundong Li, P Richard Hahn, and Huan Liu. 2020. A survey of learning causality with data: Problems and methods. ACM Computing Surveys (CSUR) 53, 4 (2020), 1--37.
[15]
Fabian Hadiji, Rafet Sifa, Anders Drachen, Christian Thurau, Kristian Kersting, and Christian Bauckhage. 2014. Predicting player churn in the wild. In 2014 IEEE Conference on Computational Intelligence and Games. IEEE, 1--8.
[16]
Jacob Haspiel, Na Du, Jill Meyerson, Lionel P Robert Jr, Dawn Tilbury, X Jessie Yang, and Anuj K Pradhan. 2018. Explanations and expectations: Trust building in automated vehicles. In Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 119--120.
[17]
Shin-Yuan Hung, David C Yen, and Hsiu-Yu Wang. 2006. Applying data mining to telecom churn management. Expert Systems with Applications 31, 3 (2006), 515--524.
[18]
Dominik Janzing, Lenon Minorics, and Patrick Blöbaum. 2020. Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics. PMLR, 2907--2916.
[19]
Emilie Kaufmann and Shivaram Kalyanakrishnan. 2013. Information complexity in bandit subset selection. In Conference on Learning Theory. 228--251.
[20]
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems. 3146--3154.
[21]
Abbas Keramati and Seyed MS Ardabili. 2011. Churn analysis for an Iranian mobile operator. Telecommunications Policy 35, 4 (2011), 344--356.
[22]
Kyoungok Kim, Chi-Hyuk Jun, and Jaewook Lee. 2014. Improved churn prediction in telecommunication industry by analyzing a large network. Expert Systems with Applications 41, 15 (2014), 6575--6584.
[23]
EunJo Lee, Yoonjae Jang, Du-Mim Yoon, JiHoon Jeon, Sung-il Yang, SangKwang Lee, Dae-Wook Kim, Pei Pei Chen, Anna Guitart, Paul Bertens, et al . 2018. Game data mining competition on churn prediction and survival analysis using com- mercial game log data. IEEE Transactions on Games (2018).
[24]
Jiayu Li, Hongyu Lu, Chenyang Wang, Weizhi Ma, Min Zhang, Xiangyu Zhao, Wei Qi, Yiqun Liu, and Shaoping Ma. 2021. A Difficulty-Aware Framework for Churn Prediction and Intervention in Games. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 943--952.
[25]
Xi Liu, Muhe Xie, Xidao Wen, Rui Chen, Yong Ge, Nick Duffield, and Na Wang. 2020. Micro-and macro-level churn analysis of large-scale mobile games. Knowl- edge and Information Systems 62, 4 (2020), 1465--1496.
[26]
Scott M Lundberg, Gabriel G Erion, and Su-In Lee. 2018. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888 (2018).
[27]
Morteza Mashayekhi and Robin Gras. 2017. Rule extraction from decision trees ensembles: new algorithms based on heuristic search and sparse group lasso methods. International Journal of Information Technology & Decision Making 16, 06 (2017), 1707--1727.
[28]
George A Miller. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review 63, 2 (1956), 81.
[29]
Milo? Milo?evi?, Nenad ?ivi?, and Igor Andjelkovi?. 2017. Early churn predic- tion with personalized targeting in mobile social games. Expert Systems with Applications 83 (2017), 326--332.
[30]
Julie Moeyersoms and David Martens. 2015. Including high-cardinality attributes in predictive models: A case study in churn prediction in the energy sector. Decision support systems 72 (2015), 72--81.
[31]
Jean Jacques Ohana, Steve Ohana, Eric Benhamou, David Saltiel, and Beatrice Guez. 2021. Explainable AI (XAI) Models Applied to the Multi-agent Environ- ment of Financial Markets. In International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. Springer, 189--207.
[32]
África Periáñez, Alain Saas, Anna Guitart, and Colin Magne. 2016. Churn pre- diction in mobile social games: Towards a complete assessment using survival ensembles. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 564--573.
[33]
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. 2017. Elements of causal inference: foundations and learning algorithms. The MIT Press.
[34]
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Doro- gush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. In Advances in Neural Information Processing Systems. 6638--6648.
[35]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1135--1144.
[36]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High- precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
[37]
Karsten Rothmeier, Nicolas Pflanzl, Joschka A Hüllmann, and Mike Preuss. 2020. Prediction of player churn and disengagement based on user activity data of a freemium online strategy game. IEEE Transactions on Games 13, 1 (2020), 78--88.
[38]
Julian Runge, Peng Gao, Florent Garcin, and Boi Faltings. 2014. Churn predic- tion for high-value players in casual social games. In 2014 IEEE conference on Computational Intelligence and Games. IEEE, 1--8.
[39]
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning im- portant features through propagating activation differences. In International Conference on Machine Learning. PMLR, 3145--3153.
[40]
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wat- tenberg. 2017. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017).
[41]
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 3319--3328.
[42]
Sarah Tan, Rich Caruana, Giles Hooker, Paul Koch, and Albert Gordo. 2018. Learning global additive explanations for neural nets using model distillation. arXiv preprint arXiv:1801.08640 (2018).
[43]
Jianrong Tao, Yu Xiong, Shiwei Zhao, Runze Wu, Xudong Shen, Tangjie Lyu, Changjie Fan, Zhipeng Hu, Sha Zhao, and Gang Pan. 2022. Explainable AI for cheating detection and churn prediction in online games. IEEE Transactions on Games (2022).
[44]
Jianrong Tao, Yu Xiong, Shiwei Zhao, Yuhong Xu, Jianshi Lin, Runze Wu, and Changjie Fan. 2020. XAI-Driven Explainable Multi-view Game Cheating Detec- tion. In 2020 IEEE Conference on Games (CoG). IEEE, 144--151.
[45]
Markus Viljanen, Antti Airola, Jukka Heikkonen, and Tapio Pahikkala. 2017. Playtime measurement with survival analysis. IEEE Transactions on Games 10, 2 (2017), 128--138.
[46]
Meng Xi, Zhiling Luo, Naibo Wang, Jianrong Tao, Ying Li, and Jianwei Yin. 2020. A Latent Feelings-aware RNN Model for User Churn Prediction with only Behaviour data. In 2020 IEEE International Conference on Smart Data Services (SMDS). IEEE, 26--35.
[47]
Carl Yang, Xiaolin Shi, Luo Jie, and Jiawei Han. 2018. I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 914--922.
[48]
Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. 2020. A survey on causal inference. arXiv preprint arXiv:2002.02770 (2020).
[49]
Xiaobing Yu, Shunsheng Guo, Jun Guo, and Xiaorong Huang. 2011. An ex- tended support vector machine forecasting framework for customer churn in e-commerce. Expert Systems with Applications 38, 3 (2011), 1425--1430.

Cited By

View all
  • (2024)A Two-Stage Ensemble Approach for Analysis of Optimizing Customer Churn with Lime Interpretability2024 IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT)10.1109/IC2PCT60090.2024.10486713(1540-1545)Online publication date: 9-Feb-2024
  • (2024)Online newspaper subscriptions: using machine learning to reduce and understand customer churnJournal of Media Business Studies10.1080/16522354.2024.234363821:4(364-387)Online publication date: 22-Apr-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2023
5996 pages
ISBN:9798400701030
DOI:10.1145/3580305
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. a/b evaluation
  2. anchor
  3. churn analysis
  4. churn prediction
  5. explainable ai
  6. treeshap

Qualifiers

  • Research-article

Conference

KDD '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)424
  • Downloads (Last 6 weeks)13
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)A Two-Stage Ensemble Approach for Analysis of Optimizing Customer Churn with Lime Interpretability2024 IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT)10.1109/IC2PCT60090.2024.10486713(1540-1545)Online publication date: 9-Feb-2024
  • (2024)Online newspaper subscriptions: using machine learning to reduce and understand customer churnJournal of Media Business Studies10.1080/16522354.2024.234363821:4(364-387)Online publication date: 22-Apr-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media