skip to main content
research-article

Enhancing GUI Exploration Coverage of Android Apps with Deep Link-Integrated Monkey

Published: 27 June 2024 Publication History

Abstract

Mobile apps are ubiquitous in our daily lives for supporting different tasks such as reading and chatting. Despite the availability of many GUI testing tools, app testers still struggle with low testing code coverage due to tools frequently getting stuck in loops or overlooking activities with concealed entries. This results in a significant amount of testing time being spent on redundant and repetitive exploration of a few GUI pages. To address this, we utilize Android’s deep links, which assist in triggering Android intents to lead users to specific pages and introduce a deep link-enhanced exploration method. This approach, integrated into the testing tool Monkey, gives rise to Delm (Deep Link-enhanced Monkey). Delm oversees the dynamic exploration process, guiding the tool out of meaningless testing loops to unexplored GUI pages. We provide a rigorous activity context mock-up approach for triggering existing Android intents to discover more activities with hidden entrances. We conduct experiments to evaluate Delm’s effectiveness on activity context mock-up, activity coverage, method coverage, and crash detection. The findings reveal that Delm can mock up more complex activity contexts and significantly outperform state-of-the-art baselines with 27.2% activity coverage, 21.13% method coverage, and 23.81% crash detection.

References

[1]
Lazaro Clapp et al. 2016. ELLA. Retrieved from https://github.com/saswatanand/ella
[2]
Alltrails, Inc. 2024. Alltrails. Retrieved from https://play.google.com/store/apps/details?id=com.alltrails.alltrails&hl=en_US&gl=US
[3]
Google LLC. 2024. Android 10. Retrieved from https://developer.android.com/about/versions/10/highlights
[4]
Google LLC. 2024. Android Adb. Retrieved from https://developer.android.com/studio/command-line/adb
[5]
Google LLC. 2024. Android Bundle. Retrieved from https://developer.android.com/reference/android/os/Bundle?hl=en
[6]
Google LLC. 2024. Android Context. Retrieved from https://developer.android.com/reference/android/content/Context
[7]
[8]
[11]
Wikipedia contributors. 2024. ES File Explorer. Retrieved from https://en.wikipedia.org/wiki/ES_File_Explorer
[12]
Google LLC. 2024. GoogleMap Documents. Retrieved from https://developers.google.com/maps/documentation/urls/get-started
[13]
GitHub contributors. 2024. IC3. Retrieved from https://github.com/siis/ic3
[14]
Jiwei Yan et al. 2024. IntentBench Homepage. Retrieved from https://github.com/hanada31/Fax/tree/master/IntentBench
[15]
JaCoCo contributors. 2024. JaCoCo. Retrieved from https://www.eclemma.org/jacoco/
[16]
[21]
GitHub contributors. 2024. Soot. Retrieved from https://github.com/soot-oss/soot
[23]
Emil Alégroth, Robert Feldt, and Lisa Ryrholm. 2015. Visual GUI testing in practice: Challenges, problems and limitations. Empir. Softw. Eng. 20 (2015), 694–744.
[24]
Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. 1–11.
[25]
Christophe Andrieu, Nando De Freitas, Arnaud Doucet, and Michael I. Jordan. 2003. An introduction to MCMC for machine learning. Mach. Learn. 50, 1 (2003), 5–43.
[26]
Andrea Arcuri and Lionel Briand. 2014. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Softw. Test., Verific. Reliab. 24, 3 (2014), 219–250.
[27]
Michael Auer, Andreas Stahlbauer, and Gordon Fraser. 2023. Android fuzzing: Balancing user-inputs and intents. In Proceedings of the IEEE Conference on Software Testing, Verification and Validation (ICST’23). IEEE, 37–48.
[28]
Tanzirul Azim, Oriana Riva, and Suman Nath. 2016. uLink: Enabling user-defined deep linking to app content. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. 305–318.
[29]
Young-Min Baek and Doo-Hwan Bae. 2016. Automated model-based Android GUI testing using multi-level GUI comparison criteria. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 238–249.
[30]
Tianqin Cai, Zhao Zhang, and Ping Yang. 2020. Fastbot: A multi-agent model-based test generation system: Beijing Bbytedance Network Technology Co., Ltd. In Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test. 93–96.
[31]
Qiuyuan Chen, Xin Xia, Han Hu, David Lo, and Shanping Li. 2021. Why my code summarization model does not work: Code comment improvement with category prediction. ACM Trans. Softw. Eng. Methodol. 30, 2 (2021), 1–29.
[32]
Sen Chen, Chunyang Chen, Lingling Fan, Mingming Fan, Xian Zhan, and Yang Liu. 2021. Accessible or not an empirical investigation of Android app accessibility. IEEE Trans. Softw. Eng. 48, 10 (2021), 3954–3968.
[33]
Sen Chen, Lingling Fan, Chunyang Chen, Ting Su, Wenhe Li, Yang Liu, and Lihua Xu. 2019. StoryDroid: Automated generation of storyboard for Android apps. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering (ICSE’19). IEEE, 596–607.
[34]
Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. 2015. Automated test input generation for Android: Are we there yet? In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE’15). IEEE, 429–440.
[35]
Stanislav Dashevskyi, Olga Gadyatskaya, Aleksandr Pilgun, and Yury Zhauniarovich. 2018. The influence of code coverage metrics on automated testing efficiency in Android. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS’18). Association for Computing Machinery, New York, NY, 2216–2218. Retrieved from DOI:
[36]
Zhen Dong, Marcel Böhme, Lucia Cojocaru, and Abhik Roychoudhury. 2020. Time-travel testing of Android apps. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20). IEEE, 481–492.
[37]
Omar El Zarif, Daniel Alencar Da Costa, Safwat Hassan, and Ying Zou. 2020. On the relationship between user churn and software issues. In Proceedings of the 17th International Conference on Mining Software Repositories. 339–349.
[38]
Sidong Feng, Mulong Xie, and Chunyang Chen. 2023. Efficiency matters: Speeding up automated testing with GUI rendering inference. In Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE’23). IEEE, 906–918.
[39]
Xiang Gao, Shin Hwei Tan, Zhen Dong, and Abhik Roychoudhury. 2018. Android testing via synthetic symbolic execution. In Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE’18). IEEE, 419–429.
[40]
Tianxiao Gu, Chun Cao, Tianchi Liu, Chengnian Sun, Jing Deng, Xiaoxing Ma, and Jian Lü. 2017. AimDroid: Activity-insulated multi-level automated testing for Android applications. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’17). IEEE, 103–114.
[41]
Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In Proceedings of the IEEE/ACM 41st International Conference on Software Engineering (ICSE’19). IEEE, 269–280.
[42]
Wunan Guo, Liwei Shen, Ting Su, Xin Peng, and Weiyang Xie. 2020. Improving automated GUI exploration of Android apps via static dependency analysis. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’20). IEEE, 557–568.
[43]
Johannes Hoffmann, Martin Ussath, Thorsten Holz, and Michael Spreitzenbarth. 2013. Slicing droids: Program slicing for smali code. In Proceedings of the 28th Annual ACM Symposium on Applied Computing. 1844–1851.
[44]
Cuixiong Hu and Iulian Neamtiu. 2011. Automating GUI testing for Android applications. In Proceedings of the 6th International Workshop on Automation of Software Test. 77–83.
[45]
Gang Hu, Xinhao Yuan, Yang Tang, and Junfeng Yang. 2014. Efficiently, effectively detecting mobile app bugs with appdoctor. In Proceedings of the 9th European Conference on Computer Systems. 1–15.
[46]
Han Hu, Qiuyuan Chen, and Zhaoyi Liu. 2019. Code generation from supervised code embeddings. In Proceedings of the 26th International Conference on Neural Information Processing (ICONIP’19). Springer, 388–396.
[47]
Han Hu, Ruiqi Dong, John Grundy, Thai Minh Nguyen, Huaxiao Liu, and Chunyang Chen. 2023. Automated mapping of adaptive app GUIs from phones to TVs. ACM Trans. Softw. Eng. Methodol. 33, 2, Article 47 (Dec.2023), 31 pages. DOI:DOI:
[48]
Han Hu, Yujin Huang, Qiuyuan Chen, Terry Yue Zhuo, and Chunyang Chen. 2023. A first look at on-device models in iOS apps. ACM Trans. Softw. Eng. Methodol. 33, 1, Article 26 (Nov.2023), 30 pages. DOI:DOI:
[49]
Hu Han, Zhan Haolan, Huang Yujin, and Liu Di. 2024. Pairwise GUI dataset construction between Android phones and tablets. Advances in Neural Information Processing Systems 36 (2024).
[50]
Casper S. Jensen, Mukul R. Prasad, and Anders Møller. 2013. Automated testing with targeted event sequence generation. In Proceedings of the International Symposium on Software Testing and Analysis. 67–77.
[51]
Emily Kowalczyk, Myra B. Cohen, and Atif M. Memon. 2018. Configurations in Android testing: They matter. In Proceedings of the 1st International Workshop on Advances in Mobile App Analysis. 1–6.
[52]
Duling Lai and Julia Rubin. 2019. Goal-driven exploration for Android applications. In Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE’19). IEEE, 115–127.
[53]
Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2019. Humanoid: A deep learning-based approach to automated black-box Android app testing. In Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE’19). IEEE, 1070–1073.
[54]
Ao Liu, Chenkai Guo, Naipeng Dong, Yinjie Wang, and Jing Xu. 2022. DALT: Deep activity launching test via intent-constraint extraction. In Proceedings of the IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE’22). IEEE, 482–493.
[55]
Changlin Liu and Xusheng Xiao. 2023. ProMal: Precise window transition graphs for Android via synergy of program analysis and machine learning. In Proceedings of the International Conference on Software Engineering. IEEE.
[56]
Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2023. Fill in the blank: Context-aware automated text input generation for mobile GUI testing. In Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE’22). IEEE, 1355–1367.
[57]
Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Zhilin Tian, Yuekai Huang, Jun Hu, and Qing Wang. 2024. Testing the limits: Unusual text inputs generation for mobile app crash detection with large language model. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–12.
[58]
Zhengwei Lv, Chao Peng, Zhao Zhang, Ting Su, Kai Liu, and Ping Yang. 2022. Fastbot2: Reusable automated model-based GUI testing for Android enhanced by reinforcement learning. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. 1–5.
[59]
Yun Ma, Ziniu Hu, Yunxin Liu, Tao Xie, and Xuanzhe Liu. 2018. Aladdin: Automating release of deep-link APIs on Android. In Proceedings of the World Wide Web Conference. 1469–1478.
[60]
Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for Android apps. In Proceedings of the 9th Joint Meeting on Foundations of Software Engineering. 224–234.
[61]
Riyadh Mahmood, Naeem Esfahani, Thabet Kacem, Nariman Mirzaei, Sam Malek, and Angelos Stavrou. 2012. A whitebox approach for automated security testing of Android applications on the cloud. In Proceedings of the 7th International Workshop on Automation of Software Test (AST’12). IEEE, 22–28.
[62]
Ke Mao, Mark Harman, and Yue Jia. 2016. Sappienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis. 94–105.
[63]
Ke Mao, Mark Harman, and Yue Jia. 2017. Crowd intelligence enhances automated mobile testing. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE’17). IEEE, 16–26.
[64]
Nadim Nachar et al. 2008. The mann-whitney U: A test for assessing whether two independent samples come from the same distribution. Tutorials in Quantitative Methods for Psychology 4, 1 (2008), 13–20.
[65]
Ana Milanova, Atanas Rountev, and Barbara G. Ryder. 2005. Parameterized object sensitivity for points-to analysis for Java. ACM Trans. Softw. Eng. Methodol. 14, 1 (2005), 1–41.
[66]
Nariman Mirzaei, Sam Malek, Corina S. Păsăreanu, Naeem Esfahani, and Riyadh Mahmood. 2012. Testing Android apps through symbolic execution. ACM SIGSOFT Softw. Eng. Notes 37, 6 (2012), 1–5.
[67]
Damien Octeau, Daniel Luchaup, Matthew Dering, Somesh Jha, and Patrick McDaniel. 2015. Composite constant propagation: Application to Android inter-component communication analysis. In Proceedings of the IEEE/ACM 37th IEEE International Conference on Software Engineering. IEEE, 77–88.
[68]
Damien Octeau, Patrick McDaniel, Somesh Jha, Alexandre Bartel, Eric Bodden, Jacques Klein, and Yves Le Traon. 2013. Effective Inter-Component communication mapping in Android: An essential step towards holistic security analysis. In Proceedings of the 22nd USENIX Security Symposium (USENIX Security’13). 543–558.
[69]
Minxue Pan, An Huang, Guoxin Wang, Tian Zhang, and Xuandong Li. 2020. Reinforcement learning based curiosity-driven testing of Android applications. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 153–164.
[70]
Strategic Planning. 2002. The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology.
[71]
Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering. 245–256.
[72]
Ting Su, Jue Wang, and Zhendong Su. 2021. Benchmarking automated GUI testing for Android against real-world bugs. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 119–130.
[73]
Ting Su, Yichen Yan, Jue Wang, Jingling Sun, Yiheng Xiong, Geguang Pu, Ke Wang, and Zhendong Su. 2021. Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs. Proc. ACM Program. Lang. 5, OOPSLA (2021), 1–31.
[74]
Yutian Tang, Yulei Sui, Haoyu Wang, Xiapu Luo, Hao Zhou, and Zhou Xu. 2020. All your app links are belong to us: Understanding the threats of instant apps based attacks. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 914–926.
[75]
Cong Tian, Congli Xia, and Zhenhua Duan. 2018. Poster: Android inter-component communication analysis with intent revision. In Proceedings of the IEEE/ACM 40th International Conference on Software Engineering (ICSE’18). IEEE, 254–255.
[76]
Yuan Tian, Meiyappan Nagappan, David Lo, and Ahmed E. Hassan. 2015. What are the characteristics of high-rated apps? A case study on free Android applications. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’15). IEEE, 301–310.
[77]
Jue Wang, Yanyan Jiang, Ting Su, Shaohua Li, Chang Xu, Jian Lu, and Zhendong Su. 2022. Detecting non-crashing functional bugs in Android apps via deep-state differential analysis. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 434–446.
[78]
Wenyu Wang, Dengfeng Li, Wei Yang, Yurui Cao, Zhenwen Zhang, Yuetang Deng, and Tao Xie. 2018. An empirical study of Android test generation tools in industrial cases. In Proceedings of the 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE’18). IEEE, 738–748.
[79]
Wenyu Wang, Wei Yang, Tianyin Xu, and Tao Xie. 2021. Vet: Identifying and avoiding UI exploration tarpits. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 83–94.
[80]
Jiwei Yan, Hao Liu, Linjie Pan, Jun Yan, Jian Zhang, and Bin Liang. 2020. Multiple-entry testing of Android applications by constructing activity launching contexts. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering (ICSE’20). IEEE, 457–468.
[81]
Shengqian Yang, Haowei Wu, Hailong Zhang, Yan Wang, Chandrasekar Swaminathan, Dacong Yan, and Atanas Rountev. 2018. Static window transition graphs for Android. Autom. Softw. Eng. 25, 4 (2018), 833–873.
[82]
Lv Zhengwei, Peng Chao, Zhang Zhao, Su Ting, Liu Kai, and Yang Ping. 2022. Fastbot2: Reusable automated model-based GUI testing for Android enhanced by reinforcement learning. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE’22).

Index Terms

  1. Enhancing GUI Exploration Coverage of Android Apps with Deep Link-Integrated Monkey

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Software Engineering and Methodology
      ACM Transactions on Software Engineering and Methodology  Volume 33, Issue 6
      July 2024
      951 pages
      EISSN:1557-7392
      DOI:10.1145/3613693
      • Editor:
      • Mauro Pezzé
      Issue’s Table of Contents

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 June 2024
      Online AM: 13 May 2024
      Accepted: 02 May 2024
      Revised: 30 April 2024
      Received: 17 October 2023
      Published in TOSEM Volume 33, Issue 6

      Check for updates

      Author Tags

      1. GUI
      2. android GUI testing
      3. android app exploration
      4. app analysis

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 322
        Total Downloads
      • Downloads (Last 12 months)322
      • Downloads (Last 6 weeks)33
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      Full Text

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media