skip to main content
10.1145/3356250.3360038acmconferencesArticle/Chapter ViewAbstractPublication PagessensysConference Proceedingsconference-collections
research-article

DeepAPP: a deep reinforcement learning framework for mobile application usage prediction

Published: 10 November 2019 Publication History

Abstract

This paper aims to predict the apps a user will open on her mobile device next. Such an information is essential for many smartphone operations, e.g., app pre-loading and content pre-caching, to save mobile energy. However, it is hard to build an explicit model that accurately depicts the affecting factors and their affecting mechanism of time-varying app usage behavior. This paper presents a deep reinforcement learning framework, named as DeepAPP, which learns a model-free predictive neural network from historical app usage data. Meanwhile, an online updating strategy is designed to adapt the predictive network to the time-varying app usage behavior. To transform DeepAPP into a practical deep reinforcement learning system, several challenges are addressed by developing a context representation method for complex contextual environment, a general agent for overcoming data sparsity and a lightweight personalized agent for minimizing the prediction time. Extensive experiments on a large-scale anonymized app usage dataset reveal that DeepAPP provides high accuracy (precision 70.6% and recall of 62.4%) and reduces the prediction time of the state-of-the-art by 6.58×. A field experiment of 29 participants also demonstrates DeepAPP can effectively reduce time of loading apps.

References

[1]
Tingxin Yan, David Chu, Deepak Ganesan, Aman Kansal, and Jie Liu. Fast app launching for mobile devices using predictive user context. In ACM MobiSys, 2012.
[2]
Xu Ye, Lin Mu, Lu Hong, Giuseppe Cardone, Nicholas Lane, Zhenyu Chen, Andrew Campbell, and Tanzeem Choudhury. Preference, context and communities:a multi-faceted approach to predicting smartphone app usage patterns. In ISWC, 2013.
[3]
David T. Nguyen, Gang Zhou, Guoliang Xing, Xin Qi, Zijiang Hao, Ge Peng, and Qing Yang. Reducing smartphone application delay through read/write isolation. In ACM MobiSys, 2015.
[4]
Abhinav Parate, Matthias BÃűhmer, David Chu, Deepak Ganesan, and Benjamin M. Marlin. Practical prediction and prefetch for faster access to applications on mobile phones. In ACM Ubicomp, 2013.
[5]
Paul Baumann and Silvia Santini. Every byte counts: Selective prefetching for mobile applications. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2):6, 2017.
[6]
Yichuan Wang, Xin Liu, David Chu, and Yunxin Liu. Earlybird: Mobile prefetching of social network feeds via content preference mining and usage pattern analysis. In ACM MobiHoc, 2015.
[7]
Joohyun Lee, Kyunghan Lee, Euijin Jeong, Jaemin Jo, and Ness B. Shroff. Context-aware application scheduling in mobile systems: what will users do and not do next? In ACM Ubicomp, 2016.
[8]
Hong Cao and Miao Lin. Mining smartphone data for app usage prediction and recommendations: A survey. Pervasive and Mobile Computing, 37:1--22, 2017.
[9]
Nagarajan Natarajan, Donghyuk Shin, and Inderjit S. Dhillon. Which app will you use next? collaborative filtering with interactional context. In ACM RecSys, 2013.
[10]
Hengshu Zhu, Huanhuan Cao, Enhong Chen, Hui Xiong, and Jilei Tian. Exploiting enriched contextual information for mobile app classification. In ACM CIKM, 2012.
[11]
Ke Huang, Chunhui Zhang, Xiaoxiao Ma, and Guanling Chen. Predicting mobile application usage using contextual information. In ACM Ubicomp, 2012.
[12]
Choonsung Shin, Jin Hyuk Hong, and Anind K. Dey. Understanding and prediction of mobile application usage for smart phones. In ACM Ubicomp, 2012.
[13]
Zhung Xun Liao, Shou Chung Li, Wen Chih Peng, Philip S. Yu, and Te Chuan Liu. On the feature discovery for app usage prediction in smartphones. In IEEE ICDM, 2014.
[14]
Ricardo Baezayates, Di Jiang, Fabrizio Silvestri, and Beverly Harrison. Predicting the next app that you are going to use. In ACM WSDM, 2015.
[15]
Xinlei Chen, Yu Wang, Jiayou He, Shijia Pan, Yong Li, and Pei Zhang. CAP: Context-aware app usage prediction with heterogeneous graph embedding. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(1):4, 2019.
[16]
Shijian Xu, Wenzhong Li, Xiao Zhang, Songcheng Gao, Tong Zhan, Yongzhu Zhao, Wei-wei Zhu, and Tianzi Sun. Predicting smartphone app usage with recurrent neural networks. In WASA, 2018.
[17]
Vassilis Kostakos, Denzil Ferreira, Jorge Goncalves, and Simo Hosio. Modelling smartphone usage: a markov state transition model. In ACM UbiComp, 2016.
[18]
V Mnih, K Kavukcuoglu, D Silver, A. A. Rusu, J Veness, M. G. Bellemare, A Graves, M Riedmiller, A. K. Fidjeland, and G Ostrovski. Human-level control through deep reinforcement learning. Nature, 518(7540):529, 2015.
[19]
Sicong Liu, Yingyan Lin, Zimu Zhou, Kaiming Nan, Hui Liu, and Junzhao Du. On-demand deep model compression for mobile devices: A usage-driven model selection framework. In ACM MobiSys, 2018.
[20]
G. Dulac-Arnold, R. Evans, H. van Hasselt, P. Sunehag, T. Lillicrap, J. Hunt, T. Mann, T. Weber, T. Degris, and B. Coppin. Deep Reinforcement Learning in Large Discrete Action Spaces. ArXiv e-prints, 2015.
[21]
Martin Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. TensorFlow: a system for large-scale machine learning. In USENIX OSDI, 2016.
[22]
TensorFlow Lite. https://tensorflow.google.cn/lite/.
[23]
WJX. www.wjx.cn.
[24]
AMAP. https://www.amap.com.
[25]
Chen Sun, Jun Zheng, Huiping Yao, Yang Wang, and D. Frank Hsu. Apprush: Using dynamic shortcuts to facilitate application launching on mobile devices. Procedia Computer Science, 19(19):445--452, 2013.
[26]
Hengshu Zhu, Hui Xiong, Yong Ge, and Enhong Chen. Ranking fraud detection for mobile apps:a holistic view. In ACM CIKM, 2013.
[27]
David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. In ICML, 2014.
[28]
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. Computer Science, 8(6):A187, 2015.
[29]
R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. Machine Learning, 8(3-4):225--227, 1992.
[30]
Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint ℓ2, 1-norms minimization. In NIPS, 2010.
[31]
Sibren Isaacman, Richard Becker, RamÃşn CÃąceres, Stephen Kobourov, Margaret Martonosi, James Rowland, and Alexander Varshavsky. Identifying important places in people's lives from cellular network data. In IEEE PerCom, 2011.
[32]
J. Cleary and I. Witten. Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications, 32(4):396--402, 1984.
[33]
Bjornar Larsen and Chinatsu Aone. Fast and effective text mining using linear-time document clustering. In ACM SIGKDD, 1999.
[34]
Gerald Albaum. The likert scale revisited. Market Research Society. Journal., 39(2):1--21, 1997.
[35]
Accubattery. https://www.accubatteryapp.com/.
[36]
Chanmin Yoon, Dongwon Kim, Wonwoo Jung, Chulkoo Kang, and Hojung Cha. AppScope: application energy metering framework for android smartphones using kernel activity monitoring. In USENIX ATC, 2012.
[37]
Zhung Xun Liao, Yi Chin Pan, Wen Chih Peng, and Po Ruey Lei. On mining mobile apps usage behavior for predicting apps usage in smartphones. In ACM CIKM, 2013.
[38]
Xiaomeng Chen, Abhilash Jindal, Ning Ding, Yu Charlie Hu, Maruti Gupta, and Rath Vannithamby. Smartphone background activities in the wild:origin, energy drain, and optimization. In ACM MOBICOM, 2015.
[39]
Gang Huang, Mengwei Xu, Felix Xiaozhu Lin, Yunxin Liu, Yun Ma, Saumay Pushp, and Xuanzhe Liu. Shuffledog: Characterizing and adapting user-perceived latency of android apps. IEEE Transactions on Mobile Computing, 16(10):2913--2926, 2017.
[40]
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized Experience Replay. ArXiv e-prints, 2015.
[41]
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In ICML, 2016.
[42]
Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. DRN: A deep reinforcement learning framework for news recommendation. In WWW, 2018.
[43]
Teng Li, Zhiyuan Xu, Jian Tang, and Yanzhi Wang. Model-free control for distributed stream data processing using deep reinforcement learning. In VLDB, 2018.
[44]
Chen Li, Lingys Justinas, Chen Kai, and Liu Feng. AuTO: Scaling deep reinforcement learning for datacenter-scale automatic traffic optimization. In ACM SIGCOMM, 2018.
[45]
Xianzhong Ding, Wan Du, and Alberto Cerpa. OCTOPUS: Deep reinforcement learning for holistic smart building control. In ACM BuildSys, 2019.
[46]
Zhiyuan Xu, Jian Tang, Jingsong Meng, Weiyi Zhang, Yanzhi Wang, Chi Harold Liu, and Dejun Yang. Experience-driven networking: A deep reinforcement learning based approach. In IEEE INFOCOM, 2018.
[47]
Zhenfeng Shao, Wenjing Wu, Zhongyuan Wang, Wan Du, and Chengyuan Li. Seaships: A large-scale precisely annotated dataset for ship detection. IEEE Transactions on Multimedia, 20(10):2593--2604, 2018.
[48]
Hongyi Yao, Gyan Ranjan, Alok Tongaonkar, Yong Liao, and Zhuoqing Morley Mao. SAMPLES:self adaptive mining of persistent lexical snippets for classifying mobile application traffic. In ACM MobiCom, 2015.
[49]
Zhihao Shen, Wan Du, Xi Zhao, and Jianhua Zou. Retrieving similar trajectories from cellular data at city scale. arXiv preprint arXiv:1907.12371, 2019.
[50]
Donghan Yu, Yong Li, Fengli Xu, Pengyu Zhang, and Vassilis Kostakos. Smartphone app usage prediction using points of interest. In ACM Ubicomp, 2018.
[51]
Zhen Tu, Runtong LI, Yong Li, Gang Wang, Di Wu, Pan Hui, Li Su, and Jin Depeng. Your apps give you away: Distinguishing mobile users by their app usage fingerprints. In ACM Ubicomp, 2018.
[52]
Huandong Wang, Chen Gao, Yong Li, Zhili Zhang, and Depeng Jin. From fingerprint to footprint: Revealing physical world privacy leakage by cyberspace cookie logs. In ACM CIKM, 2017.
[53]
Deren Li and Zhenfeng Shao. The new era for geo-information. Science in China Series F: Information Sciences, 52(7):1233--1242, 2009.

Cited By

View all
  • (2025)Exploring the Adverse Impact of Smartphone Use on Young Individuals' Self-Esteem: A Structural Equation Modeling Approach based on Five TemperamentsEngineering, Technology & Applied Science Research10.48084/etasr.936915:1(19793-19801)Online publication date: 2-Feb-2025
  • (2025)Learning Road Network Index Structure for Efficient Map MatchingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348519537:1(423-437)Online publication date: Jan-2025
  • (2025)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903265(125903)Online publication date: Mar-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SenSys '19: Proceedings of the 17th Conference on Embedded Networked Sensor Systems
November 2019
472 pages
ISBN:9781450369503
DOI:10.1145/3356250
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. app usage prediction
  2. deep reinforcement learning
  3. mobile devices
  4. neural networks

Qualifiers

  • Research-article

Conference

Acceptance Rates

Overall Acceptance Rate 198 of 990 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)78
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Exploring the Adverse Impact of Smartphone Use on Young Individuals' Self-Esteem: A Structural Equation Modeling Approach based on Five TemperamentsEngineering, Technology & Applied Science Research10.48084/etasr.936915:1(19793-19801)Online publication date: 2-Feb-2025
  • (2025)Learning Road Network Index Structure for Efficient Map MatchingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348519537:1(423-437)Online publication date: Jan-2025
  • (2025)Appformer: A novel framework for mobile app usage prediction leveraging progressive multi-modal data fusion and feature extractionExpert Systems with Applications10.1016/j.eswa.2024.125903265(125903)Online publication date: Mar-2025
  • (2024)A Low-Density Parity-Check Coding Scheme for LoRa NetworkingACM Transactions on Sensor Networks10.1145/366592820:4(1-29)Online publication date: 8-Jul-2024
  • (2024)Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the FieldACM Transactions on Sensor Networks10.1145/366218220:4(1-34)Online publication date: 8-Jul-2024
  • (2024)AttAuth: An Implicit Authentication Framework for Smartphone Users Using Multimodality DataIEEE Internet of Things Journal10.1109/JIOT.2023.331471711:4(6928-6942)Online publication date: 15-Feb-2024
  • (2024)IncreAuth: Incremental-Learning-Based Behavioral Biometric Authentication on SmartphonesIEEE Internet of Things Journal10.1109/JIOT.2023.328993511:1(1589-1603)Online publication date: 1-Jan-2024
  • (2024)Local search resource allocation algorithm for space-based backbone network in Deep Reinforcement Learning methodAd Hoc Networks10.1016/j.adhoc.2024.103575163(103575)Online publication date: Oct-2024
  • (2024)Federated privacy-preserving collaborative filtering for on-device next app predictionUser Modeling and User-Adapted Interaction10.1007/s11257-024-09395-034:4(1369-1398)Online publication date: 28-Mar-2024
  • (2023)TPP: Accelerate Application Launch via Two-Phase Prefetching on Smartphone2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10136908(1-6)Online publication date: Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media