Abstract
In recent years, more and more systems have been designed to affect users’ decisions for realizing certain system goals. However, most of these systems only focus on affecting users’ short-term or one-off behaviors, while ignoring the maintenance of users’ long-term engagement. In this light, we intend to design a novel approach which focuses on incentivizing users’ long-term engagement. In this paper, inspired by the use of Markov Decision Process (MDP), we first formally model the process of a user’s decision-making under long-term incentives. Subsequently, we propose the MDP-based Incentive Estimation (MDP-IE) approach for determining the value of an incentive and the requirement of obtaining that incentive. Experimental results demonstrate that the proposed approach can effectively sustain users’ long-term engagement. Furthermore, the experiments also demonstrate that incentivizing users’ long-term engagement is more beneficial than one-off or short-term approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bellman, R.: A Markovian decision process. J. Math. Mech. 6, 679–684 (1957)
Bonabeau, E.: Agent-based modeling: methods and techniques for simulating human systems. Proc. Nat. Acad. Sci. 99(3), 7280–7287 (2002)
Gan, X., Wang, X., Niu, W., Hang, G., Tian, X., Wang, X., Xu, J.: Incentivize multi-class crowd labeling under budget constraint. IEEE J. Sel. Areas Commun. 35(4), 893–905 (2017)
Homans, G.C.: Social Behavior: Its Elementary Forms. Harcourt Brace Jovanovich, San Diego (1974)
Iversen, E.B., Morales, J.M., Madsen, H.: Optimal charging of an electric vehicle using a Markov decision process. Appl. Energy 123, 1–12 (2014)
Ksentini, A., Taleb, T., Chen, M.: A Markov decision process-based service migration procedure for follow me cloud. In: 2014 IEEE International Conference on Communications (ICC), pp. 1350–1354 (2014)
Li, W., Bai, Q., Zhang, M., Nguyen, T.D.: Automated influence maintenance in social networks: an agent-based approach. IEEE Trans. Knowl. Data Eng. (2018). https://doi.org/10.1109/TKDE.2018.2867774
Liu, Y.: The long-term impact of loyalty programs on consumer purchase behavior and loyalty. J. Mark. 71(4), 19–35 (2007)
Launch Marketing: What are your short-and long-term marketing strategies (2015)
Sengvong, S., Bai, Q.: Persuasive public-friendly route recommendation with flexible rewards. In: 2017 IEEE International Conference on Agents, pp. 109–114 (2017)
Singla, A., Krause, A.: Truthful incentives in crowdsourcing tasks using regret minimization mechanisms. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 1167–1178 (2013)
Singla, A., Santoni, M., Bartók, G., Mukerji, P., Meenen, M., Krause, A.: Incentivizing users for balancing bike sharing systems. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 723–729 (2015)
Terefe, M.B., Lee, H., Heo, N., Fox, G.C., Oh, S.: Energy-efficient multisite offloading policy using markov decision process for mobile cloud computing. Pervasive Mob. Comput. 27, 75–89 (2016)
Tran-Thanh, L., Chapman, A., de Cote, E.M., Rogers, A., Jennings, N.R.: Epsilon-first policies for budget-limited multi-armed bandits. In: Proceedings of Twenty-Fourth AAAI Conference on Artificial Intelligence, pp. 1211–1216 (2010)
Tran-Thanh, L., Chapman, A., Rogers, A., Jennings, N.R.: Knapsack based optimal policies for budget-limited multi-armed bandits. In: Proceedings of Twenty-Sixth AAAI Conference on Artificial Intelligence, pp. 1134–1140 (2012)
Truong, N.V., Stein, S., Tran-Thanh, L., Jennings, N.R.: Adaptive incentive selection for crowdsourcing contests. In: Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems, pp. 2100–2102 (2018)
Wu, S., Bai, Q., Sengvong, S.: GreenCommute: an influence-aware persuasive recommendation approach for public-friendly commute options. J. Syst. Sci. Syst. Eng. 27(2), 250–264 (2018)
Yu, H., Miao, C., Chen, Y., Fauvel, S., Li, X., Lesser, V.R.: Algorithmic management for improving collective productivity in crowdsourcing. Sci. Rep. 7, 12541 (2017). https://doi.org/10.1038/s41598-017-12757-x
Zhao, D., Li, B., Xu, J., Hao, D., Jennings, N.R.: Selling multiple items via social networks. In: Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems, pp. 68–76 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, S., Bai, Q. (2019). Incentivizing Long-Term Engagement Under Limited Budget. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11670. Springer, Cham. https://doi.org/10.1007/978-3-030-29908-8_52
Download citation
DOI: https://doi.org/10.1007/978-3-030-29908-8_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29907-1
Online ISBN: 978-3-030-29908-8
eBook Packages: Computer ScienceComputer Science (R0)