Victor M. Magalhães Pinto
ABSTRACT
The use of daily push notifications is prevalent in many online and mobile applications to enhance and maintain user engagement. Push notifications are often used in customer relationship management (CRM) campaigns to promote engagement, and frequently a customer is subjected to several on a daily basis and many at the same time. This often results in multiple notifications being scheduled to a user simultaneously. Also, in online apps, push notifications can trigger new orders during various shifts throughout the day for each user. This paper presents a complete framework of push notification modeling that takes into account the human-in-the-loop aspect of the problem, mixing up modeling with business decisions. The model structure is based on a two-tower deep learning model to rank push notifications based on their relevance to users, utilizing push metadata and user features. It also analyzes the causal impact of sending push notifications during each shift of the day. We use it to successfully optimize more than 100 million daily push notifications on the food delivery app iFood, resulting in increased orders and reduced average push notifications per user.
- Charu C Aggarwal 2016. Recommender Systems. Springer. https://doi.org/10.1007/978-3-319-29659-3Google Scholar
Cross Ref
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.Google Scholar
- Gautam Chauhan, Dhruva Vatsa Mishra, M Farida Begam, 2019. Customer-Aware Recommender System for Push Notifications in an e-commerce Environment. In 2019 Global Conference for Advancement in Technology (GCAT). IEEE, 1–7. https://doi.org/10.1109/GCAT47503.2019.8978330Google ScholarCross Ref
- Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7–10. https://doi.org/10.1145/2988450.2988454Google ScholarDigital Library
- Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198. https://doi.org/10.1145/2959100.2959190Google ScholarDigital Library
- Felipe Ferreira, Daniele R Souza, Igor Moura, Matheus Barbieri, and Hélio CV Lopes. 2020. Investigating multimodal features for video recommendations at globoplay. In Proceedings of the 14th ACM Conference on Recommender Systems. 571–572. https://doi.org/10.1145/3383313.3411553Google ScholarDigital Library
- Ronald Aylmer Fisher. 1936. Design of experiments. British Medical Journal 1, 3923 (1936), 554.Google ScholarCross Ref
- Diana Gavilan and Gema Martinez-Navarro. 2022. Exploring user’s experience of push notifications: a grounded theory approach. Qualitative Market Research: An International Journal (2022). https://doi.org/10.1108/QMR-05-2021-0061Google ScholarCross Ref
- Merve Gençer, Gökhan Bilgin, Özgür Zan, and Tansel Voyvodaoğlu. 2013. A new framework for increasing user engagement in mobile applications using machine learning techniques. In Design, User Experience, and Usability. Web, Mobile, and Product Design: Second International Conference, DUXU 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part IV 2. Springer, 651–659. https://doi.org/10.1007/978-3-642-39253-5_72Google ScholarDigital Library
- Pierre Gutierrez and Jean-Yves Gérardy. 2017. Causal inference and uplift modelling: A review of the literature. In International conference on predictive applications and APIs. PMLR, 1–13.Google Scholar
- Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182. https://doi.org/10.1145/3038912.3052569Google ScholarDigital Library
- Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learning representations for counterfactual inference. In International conference on machine learning. PMLR, 3020–3029.Google Scholar
- Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing (2nd Edition). Prentice-Hall, Inc., USA.Google Scholar
- Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. 2017. Causal effect inference with deep latent-variable models. Advances in neural information processing systems 30 (2017).Google Scholar
- Adithya Madhusoodanan, Anand Kumar, Kieran Fraser, and Bilal Yousuf. 2020. Machine Learning Approach to Manage Adaptive Push Notifications for Improving User Experience. In MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 488–493. https://doi.org/10.1145/3448891.3448956Google ScholarDigital Library
- Conor O’Brien, Huasen Wu, Shaodan Zhai, Dalin Guo, Wenzhe Shi, and Jonathan J Hunt. 2022. Should I send this notification? Optimizing push notifications decision making by modeling the future. arXiv preprint arXiv:2202.08812 (2022). https://doi.org/10.48550/arXiv.2202.08812Google ScholarCross Ref
- Tadashi Okoshi, Kota Tsubouchi, and Hideyuki Tokuda. 2019. Real-world product deployment of adaptive push notification scheduling on smartphones. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2792–2800. https://doi.org/10.1145/3292500.3330732Google ScholarDigital Library
- Kaare Brandt Petersen, Michael Syskind Pedersen, 2008. The matrix cookbook. Technical University of Denmark 7, 15 (2008), 510.Google Scholar
- Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training. (2018).Google Scholar
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.Google Scholar
- Mark J Schervish and Morris H DeGroot. 2012. Probability and statistics. Pearson Education.Google Scholar
- Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning. PMLR, 3076–3085.Google Scholar
- Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting Neural Networks for the Estimation of Treatment Effects. In Advances in Neural Information Processing Systems, H Wallach, H Larochelle, A Beygelzimer, F Alché-Buc, E Fox, and R Garnett (Eds.). Vol. 32. Curran Associates, Inc.https://proceedings.neurips.cc/paper_files/paper/2019/file/8fb5f8be2aa9d6c64a04e3ab9f63feee-Paper.pdfGoogle Scholar
- Tian Wang, Yuri M Brovman, and Sriganesh Madhvanath. 2021. Personalized embedding-based e-commerce recommendations at ebay. arXiv preprint arXiv:2102.06156 (2021). https://doi.org/10.48550/arXiv.2102.06156Google ScholarCross Ref
- Kevin P Yancey and Burr Settles. 2020. A sleeping, recovering bandit algorithm for optimizing recurring notifications. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3008–3016. https://doi.org/10.1145/3394486.3403351Google ScholarDigital Library
Index Terms
- Smart Notifications – An ML-based Framework to Boost User Engagement
Recommendations
DaPanda: detecting aggressive push notifications in Android apps
ASE '19: Proceedings of the 34th IEEE/ACM International Conference on Automated Software EngineeringMobile push notifications have been widely used in mobile platforms to deliver all sorts of information to app users. Although it offers great convenience for both app developers and mobile users, this feature was frequently reported to serve malicious ...
Contextual Push Notifications on Mobile Devices: A Pre-study on the Impact of Usage Context on User Response
Mobile Web and Intelligent Information SystemsAbstractIn a time where users are facing increasing amounts of daily push notifications, a variety of research has been conducted to find a more systematic way to deliver content to the user more efficiently. These studies aim to establish scheduled ...
Intelligent Push Notification for Converged Mobile Computing and Internet of Things
ICWS '15: Proceedings of the 2015 IEEE International Conference on Web ServicesPush notification is an important approach to distribute interesting information to users timely. With the fast development of mobile devices and mobile applications, push notification is getting more and more popular. The convergence of mobile and IoT ...
Comments