Skip to main content
SpringerLink
Log in

Iterative Android automated testing

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

With the benefits of reducing time and workforce, automated testing has been widely used for the quality assurance of mobile applications (APPs). Compared with automated testing, manual testing can achieve higher coverage in complex interactive Activities. And the effectiveness of manual testing is highly dependent on the user operation process (UOP) of experienced testers. Based on the UOP, we propose an iterative Android automated testing (IAAT) method that automatically records, extracts, and integrates UOPs to guide the test logic of the tool across the complex Activity iteratively. The feedback test results can train the UOPs to achieve higher coverage in each iteration. We extracted 50 UOPs and conducted experiments on 10 popular mobile APPs to demonstrate IAAT’s effectiveness compared with Monkey and the initial automated tests. The experimental results show a noticeable improvement in the IAAT compared with the test logic without human knowledge. Under the 60 minutes test time, the average code coverage is improved by 13.98% to 37.83%, higher than the 27.48% of Monkey under the same conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Pecorelli F, Catolino G, Ferrucci F, De Lucia A, Palomba F. Software testing and Android applications: a large-scale empirical study. Empirical Software Engineering, 2022, 27(2): 31

    Article  Google Scholar 

  2. Peng C, Rajan A, Cai T. CAT: change-focused android GUI testing. In: Proceedings of 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). 2021, 460–470

  3. Salehnamadi N, Alshayban A, Lin J W, Ahmed I, Branham S, Malek S. Latte: use-case and assistive-service driven automated accessibility testing framework for android. In: Proceedings of 2021 CHI Conference on Human Factors in Computing Systems. 2021, 274

  4. Ravelo-Méndez W, Escobar-Velásquez C, Linares-Vásquez M. Kraken: a framework for enabling multi-device interaction-based testing of Android APPs. Science of Computer Programming, 2021, 206: 102627

    Article  Google Scholar 

  5. Noh M J, Lee K T. An analysis of the relationship between quality and user acceptance in smartphone APPs. Information Systems and eBusiness Management, 2016, 14(2): 273–291

    Google Scholar 

  6. Sun S, Fu X, Ruan H, Du X, Luo B, Guizani M. Real-time behavior analysis and identification for android application. IEEE Access, 2018, 6: 38041–38051

    Article  Google Scholar 

  7. Amalfitano D, Fasolino A R, Tramontana P, De Carmine S, Memon A M. Using GUI ripping for automated testing of Android applications. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. 2012, 258–261

  8. Huang R, Zhang Q, Towey D, Sun W, Chen J. Regression test case prioritization by code combinations coverage. Journal of Systems and Software, 2020, 169: 110712

    Article  Google Scholar 

  9. Cai G, Su Q, Hu Z. Automated test case generation for path coverage by using grey prediction evolution algorithm with improved scatter search strategy. Engineering Applications of Artificial Intelligence, 2021, 106: 104454

    Article  Google Scholar 

  10. Liu Z, Chen C, Wang J, Huang Y, Hu J, Wang Q. Guided bug crush: assist manual GUI testing of android APPs via hint moves. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022, 557

  11. Yasin H N, Ab Hamid S H, Yusof R J R. DroidbotX: test case generation tool for android applications using Q-learning. Symmetry, 2021, 13(2): 310

    Article  Google Scholar 

  12. Li L, Bissyandé T F, Papadakis M, Rasthofer S, Bartel A, Octeau D, Klein J, Traon L. Static analysis of android APPs: a systematic literature review. Information and Software Technology, 2017, 88: 67–95

    Article  Google Scholar 

  13. Kong P, Li L, Gao J, Liu K, Bissyandé T F, Klein J. Automated testing of android APPs: a systematic literature review. IEEE Transactions on Reliability, 2019, 68(1): 45–66

    Article  Google Scholar 

  14. Méndez-Porras A, Quesada-López C, Jenkins M. Automated testing of mobile applications: a systematic map and review. In: Proceedings of the XVIII IberoAmerican Conference on Software Engineering. 2015, 195

  15. Pilgun A, Gadyatskaya O, Zhauniarovich Y, Dashevskyi S, Kushniarou A, Mauw S. Fine-grained code coverage measurement in automated black-box android testing. ACM Transactions on Software Engineering and Methodology, 2020, 29(4): 23

    Article  Google Scholar 

  16. Liu S. Improvement and implementation of android Robotium automated testing framework system. Southeast University, Dissertation, 2017

  17. Geng Z. Study and improvement of android automatic testing. Beijing University of Posts and Telecommunications, Dissertation, 2017

  18. Choudhary S R, Gorla A, Orso A. Automated test input generation for android: are we there yet? (E). In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering. 2015, 429–440

  19. Mirzaei N, Garcia J, Bagheri H, Sadeghi A, Malek S. Reducing combinatorics in GUI testing of android applications. In: Proceedings of the 38th IEEE/ACM International Conference on Software Engineering. 2016, 559–570

  20. Hu Y, Neamtiu I, Alavi A. Automatically verifying and reproducing event-based races in Android APPs. In: Proceedings of the 25th International Symposium on Software Testing and Analysis. 2016, 377–388

  21. Clapp L, Bastani O, Anand S, Aiken A. Minimizing GUI event traces. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2016, 422–434

  22. Heiskanen H, Maunumaa M, Katara M. A test process improvement model for automated test generation. In: Proceedings of the 13th International Conference on Product-Focused Software Process Improvement. 2012, 17–31

  23. Yu S, Fang C, Feng Y, Zhao W, Chen Z. LIRAT: layout and image recognition driving automated mobile testing of cross-platform. In: Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering. 2019, 1066–1069

  24. Grano G, Ciurumelea A, Panichella S, Palomba F, Gall H C. Exploring the integration of user feedback in automated testing of Android applications. In: Proceedings of the 25th IEEE International Conference on Software Analysis, Evolution and Reengineering. 2018, 72–83

  25. Gu Y, Shi J L. Generality for Technology of Software Testing. Beijing: Tsinghua University Press, 2004

    Google Scholar 

  26. Mahmood R, Mirzaei N, Malek S. EvoDroid: segmented evolutionary testing of Android APPs. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2014, 599–609

  27. Su T, Meng G, Chen Y, Wu K, Yang W, Yao Y, Pu G, Liu Y, Su Z. Guided, stochastic model-based GUI testing of Android APPs. In: Proceedings of the 11th Joint Meeting on Foundations of Software Engineering. 2017, 245–256

  28. Mao K, Harman M, Jia Y. Sapienz: multi-objective automated testing for Android applications. In: Proceedings of the 25th International Symposium on Software Testing and Analysis. 2016, 94–105

  29. Behrang F, Orso A. AppTestMigrator: a tool for automated test migration for Android APPs. In: Proceedings of the 42nd IEEE/ACM International Conference on Software Engineering: Companion Proceedings. 2020, 17–20

  30. Chen S, Fan L, Chen C, Su T, Li W, Liu Y, Xu L. StoryDroid: automated generation of storyboard for android APPs. In: Proceedings of the 41st IEEE/ACM International Conference on Software Engineering. 2019, 596–607

  31. Fan L, Su T, Chen S, Meng G, Liu Y, Xu L, Pu G. Efficiently manifesting asynchronous programming errors in Android APPs. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. 2018, 486–497

  32. Pan M, Huang A, Wang G, Zhang T, Li X. Reinforcement learning based curiosity-driven testing of Android applications. In: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 2020, 153–164

  33. Dong Z, Böhme M, Cojocaru L, Roychoudhury A. Time-travel testing of android APPs. In: Proceedings of the 42nd IEEE/ACM International Conference on Software Engineering. 2020, 481–492

  34. Zhang X, Chen Z, Fang C, Liu Z. Guiding the crowds for Android testing. In: Proceedings of the 38th International Conference on Software Engineering Companion. 2016, 752–753

  35. Meng C. A research on android test automation technology based on dependency injection. Nanjing University, Dissertation, 2017

  36. Mao K, Harman M, Jia Y. Crowd intelligence enhances automated mobile testing. In: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. 2017, 16–26

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. 62141215); the National Key R&D Program of China: R&D and Application of Integrated Crowdsourcing Test Service Platform for Information Products and Technology Services (2018YFB1403400); and the Science, Technology and Innovation Commission of Shenzhen Municipality (CJGJZD20200617103001003).

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, China

    Yi Zhong, Mengyu Shi, Youran Xu, Chunrong Fang & Zhenyu Chen

  2. School of Big Data and Computer Science, Chongqing College of Mobile Communication, Chongqing, 400065, China

    Yi Zhong

Authors
  1. Yi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mengyu Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youran Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunrong Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenyu Chen.

Additional information

Yi Zhong received the BS degree in Computer Application Technology from Chongqing University of Posts and Telecommunications, China in 2009 and the MS degree in Computer Application Technology from Chongqing University of Posts and Telecommunications, China in 2013. She is working toward the PhD degree in Software Engineering of Nanjing University, China. Her research interests include artificial intelligence and software testing.

Mengyu Shi received the BS degree in Computer Science and Technology from Southwest University, China in 2021. She is currently working toward the MS degree in Software Engineering in Nanjing University, China. Her research interest is software testing.

Youran Xu received the BS degree in Software Engineering from Soochow University, China, Business College in 2021 and the MS degree in Software Engineering from Nanjing University, China. His research interest is software testing and mobile application testing.

Chunrong Fang, the Research Assistant of Software Institute, Nanjing University, China. His teaching include Foundations of Computing Systems(Freshman), Software Engineering and Computing II(Sophomore), Software Engineering and Computing III(Sophomore) and Automation Test(Junior). His research interest is Bigcode Quality and AITesting.

Zhenyu Chen, the Full Professor of Software Institute, Nanjing University, China. He is the main teacher of Statistical Methods and Data Analytics and the Software Testing: Methods and Techniques at Nanjing University, China. He has published a total of 86 papers as the first author or co-author. He is the sociate Editor of IEEE Transactions on Reliability. He is also the Contest Co-Chair in China at QRS 2018, ICST 2019, ISSTA 2019. Besides, he is the Industrial Track Co-Chair of SANER 2019, PC member of ISSRE 2018. His research interests include collective intelligence, deep learning testing and optimization, big data quality, and mobile application testing.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, Y., Shi, M., Xu, Y. et al. Iterative Android automated testing. Front. Comput. Sci. 17, 175212 (2023). https://doi.org/10.1007/s11704-022-1658-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-1658-8

Keywords

Search

Navigation