skip to main content
10.1145/3447548.3467277acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

A Difficulty-Aware Framework for Churn Prediction and Intervention in Games

Published: 14 August 2021 Publication History

Abstract

User's leaving from the system without further return, called user churn, is a severe negative signal in online games. Therefore, churn prediction and intervention are of great value for improving players' experiences and system performance. However, the problem has not been well-studied in the game scenario. Especially, some crucial factors, such as game difficulty, have not been considered for large-scale churn analysis. In this paper, a novel Difficulty-Aware Framework (DAF) for churn prediction and intervention is proposed. Firstly, a Difficulty Flow for each user is proposed, which is utilized to derive users' Personalized Perceived Difficulty during the game process. Then, a survival analysis modelD-Cox-Time is designed to model the Dynamic Influence of Perceived Difficulty on player churn intention. Finally, thePersonalized Perceived Difficulty ~(PPD) andDynamic Difficulty Influence ~(DDI) are incorporated to churn prediction and intervention. The proposed DAF framework has been specified in a real-world puzzle game as an example for churn prediction and intervention. Extensive offline experiments show significant improvements in churn prediction by introducing difficulty-related features. Besides, we conduct an online intervention system to adjust difficulty dynamically in the online game. A/B test results verify that the proposed intervention system enhances user retention and engagement significantly. To the best of our knowledge, it is the first framework in games that illustrates an in-depth understanding and leveraging dynamic and personalized perceived difficulty during game playing, which is easy to be integrated with various churn prediction and intervention models.

Supplementary Material

MP4 File (a_difficultyaware_framework_for_churn-jiayu_li-hongyu_lu-38957851-7sIg.mp4)
Presentation video for A Difficulty-Aware Framework for Churn Prediction and Intervention in Games. We model in-depth understanding of the difficulty in video games, and conduct churn prediction (offline) and intervention (online) tasks. Data and codes are available at https://github.com/THUIR/DAF-for-churn.

References

[1]
Jae-Hyeon Ahn, Sang-Pil Han, and Yung-Seop Lee. 2006. Customer churn analysis: Churn determinants and mediation effects of partial defection in the Korean mobile telecommunications service industry. Telecommunications policy 30, 10--11 (2006), 552--568.
[2]
Maria-Virginia Aponte, Guillaume Levieux, and Stephane Natkin. 2011. Measuring the level of difficulty in single player video games. Entertainment Computing 2, 4 (2011), 205--213.
[3]
Paul Bertens, Anna Guitart, and África Periáñez. 2017. Games and big data: A scalable multi-dimensional churn prediction model. In 2017 IEEE conference on computational intelligence and games (CIG). IEEE, 33--36.
[4]
Valerio Bonometti, Charles Ringer, Mark Hall, Alex R Wade, and Anders Drachen. 2019. Modelling Early User-Game Interactions for Joint Estimation of Survival Time and Churn Probability. In 2019 IEEE Conference on Games (CoG). IEEE, 1--8.
[5]
Sara Bunian, Alessandro Canossa, Randy Colvin, and Magy Seif El-Nasr. 2017. Modeling individual differences in game behavior using HMM. In AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 13.
[6]
Thomas Constant and Guillaume Levieux. 2019. Dynamic difficulty adjustment impact on players' confidence. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--12.
[7]
Thomas Constant, Guillaume Levieux, Axel Buendia, and Stéphane Natkin. 2017. From objective to subjective difficulty evaluation in video games. In IFIP Conference on Human-Computer Interaction. Springer, 107--127.
[8]
Mihaly Csikszentmihalyi and Mihaly Csikzentmihaly. 1990. Flow: The psychology of optimal experience. Vol. 1990. Harper & Row New York.
[9]
Carol S Dweck and Andrew J Elliot. 2005. Handbook of competence and motivation. Guilford Press New York.
[10]
Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. 1999. Learning to forget: Continual prediction with LSTM. (1999).
[11]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. arXiv preprint arXiv:1703.04247 (2017).
[12]
Katja Hofmann, Lihong Li, and Filip Radlinski. 2016. Online evaluation for information retrieval. FnTIR 10, 1 (2016), 1--117.
[13]
Jesper Juul. 2009. Fear of failing? the many meanings of difficulty in video games. The video game theory reader 2, 01 (2009), 2009.
[14]
John P Klein and Melvin L Moeschberger. 2006. Survival analysis: techniques for censored and truncated data. Springer Science & Business Media.
[15]
Håvard Kvamme, Ørnulf Borgan, and Ida Scheel. 2019. Time-to-Event Prediction with Neural Networks and Cox Regression. arXiv:1907.00825 (Sept. 2019). arXiv: 1907.00825.
[16]
Xi Liu, Muhe Xie, Xidao Wen, Rui Chen, Yong Ge, Nick Duffield, and Na Wang. 2020. Micro-and macro-level churn analysis of large-scale mobile games. Knowledge and Information Systems 62, 4 (2020), 1465--1496.
[17]
Derek Lomas, Kishan Patel, Jodi L Forlizzi, and Kenneth R Koedinger. 2013. Optimizing challenge in an educational game using large-scale design experiments. In SIGCHI. 89--98.
[18]
Nico JD Nagelkerke et al. 1991. A note on a general definition of the coefficient of determination. Biometrika 78, 3 (1991), 691--692.
[19]
Africa Perianez, Alain Saas, Anna Guitart, and Colin Magne. 2016. Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles. In DSAA.
[20]
Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2020. Enemy within: Long-term motivation effects of deep player behavior models for dynamic difficulty adjustment. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--10.
[21]
Hua Qin, Pei-Luen Patrick Rau, and Gavriel Salvendy. 2010. Effects of different scenarios of game difficulty on player immersion. Interacting with Computers 22, 3 (2010), 230--239.
[22]
Shaghayegh Roohi, Asko Relas, Jari Takatalo, Henri Heiskanen, and Perttu Hämäläinen. 2020. Predicting Game Difficulty and Churn Without Players. In The Annual Symposium on Computer-Human Interaction in Play. 585--593.
[23]
Karsten Rothmeier, Nicolas Pflanzl, Joschka Hüllmann, and Mike Preuss. 2020. Prediction of Player Churn and Disengagement Based on User Activity Data of a Freemium Online Strategy Game. IEEE Transactions on Games (2020).
[24]
Mehpara Saghir, Zeenat Bibi, Saba Bashir, and Farhan Hassan Khan. 2019. Churn prediction using neural network based individual and ensemble models. In 2019 16th International Bhurban Conference on Applied Sciences and Technology (IB-CAST). IEEE, 634--639.
[25]
Katie Salen, Katie Salen Tekinba?, and Eric Zimmerman. 2004. Rules of play: Game design fundamentals. MIT press.
[26]
Yoones A Sekhavat. 2017. MPRL: Multiple-Periodic Reinforcement Learning for difficulty adjustment in rehabilitation games. In 2017 IEEE 5th international conference on serious games and applications for health (SeGAH). IEEE, 1--7.
[27]
Penelope Sweetser and Peta Wyeth. 2005. GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE) 3, 3 (2005), 3--3.
[28]
Chin Hiong Tan, Kay Chen Tan, and Arthur Tay. 2011. Dynamic game difficulty scaling using adaptive behavior-based AI. IEEE Transactions on Computational Intelligence and AI in Games 3, 4 (2011), 289--301.
[29]
Qiu-Feng Wang, Mirror Xu, and Amir Hussain. 2019. Large-scale ensemble model for customer churn prediction in search ads. Cognitive Computation 11, 2 (2019), 262--270.
[30]
Artit Wangperawong, Cyrille Brun, Olav Laudy, and Rujikorn Pavasuthipaisit. 2016. Churn analysis using deep convolutional neural networks and autoencoders. arXiv preprint arXiv:1604.05377 (2016).
[31]
Su Xue, Meng Wu, John Kolen, Navid Aghdaie, and Kazi A Zaman. 2017. Dynamic difficulty adjustment for maximized engagement in digital games. In Proceedings of the 26th International Conference on World Wide Web Companion. 465--471.
[32]
Carl Yang, Xiaolin Shi, Luo Jie, and Jiawei Han. 2018. I know you'll be back: Interpretable new user clustering and churn prediction on a mobile social application. In SIGKDD. 914--922.

Cited By

View all
  • (2024)Game Difficulty Prediction Based on Facial Cues and Game PerformanceApplied Sciences10.3390/app1419877814:19(8778)Online publication date: 28-Sep-2024
  • (2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
  • (2024)Damage Optimization in Video Games: A Player-Driven Co-Creative ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642643(1-16)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
August 2021
4259 pages
ISBN:9781450383325
DOI:10.1145/3447548
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. churn intervention
  2. churn prediction
  3. game difficulty

Qualifiers

  • Research-article

Funding Sources

Conference

KDD '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)81
  • Downloads (Last 6 weeks)8
Reflects downloads up to 27 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Game Difficulty Prediction Based on Facial Cues and Game PerformanceApplied Sciences10.3390/app1419877814:19(8778)Online publication date: 28-Sep-2024
  • (2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
  • (2024)Damage Optimization in Video Games: A Player-Driven Co-Creative ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642643(1-16)Online publication date: 11-May-2024
  • (2023)Quantifying and Leveraging User Fatigue for Interventions in Recommender SystemsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592044(2293-2297)Online publication date: 19-Jul-2023
  • (2023)Measuring Item Global Residual Value for Fair RecommendationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591724(269-278)Online publication date: 19-Jul-2023
  • (2023)Characterizing Internet Card User Portraits for Efficient Churn Prediction Model DesignIEEE Transactions on Mobile Computing10.1109/TMC.2023.324120623:2(1735-1752)Online publication date: 31-Jan-2023
  • (2023)TGT: Churn Prediction in O2O Logistics with Two-tower Gated Transformer2023 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00129(744-751)Online publication date: 21-Dec-2023
  • (2023)Intelligent Verbal Interaction Methods with Non-Player Characters in Metaverse Applications2023 IEEE 5th International Conference on Advanced Information and Communication Technologies (AICT)10.1109/AICT61584.2023.10452688(67-71)Online publication date: 21-Nov-2023
  • (2022)Personalized Game Difficulty Prediction Using Factorization MachinesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545624(1-13)Online publication date: 29-Oct-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media