research-article

Reinforcement learning for Android GUI testing

Authors:

Md Khorrom Khan,

Sreedevi Koppula,

Renée BryceAuthors Info & Claims

A-TEST 2018: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

Pages 2 - 8

https://doi.org/10.1145/3278186.3278187

Published: 05 November 2018 Publication History

Abstract

This paper presents a reinforcement learning approach to automated GUI testing of Android apps. We use a test generation algorithm based on Q-learning to systematically select events and explore the GUI of an application under test without requiring a preexisting abstract model. We empirically evaluate the algorithm on eight Android applications and find that the proposed approach generates test suites that achieve between 3.31% to 18.83% better block-level code coverage than random test generation.

References

[1]

David Adamo, Renée Bryce, and Tariq M. King. 2018. Randomized Event Sequence Generation Strategies for Automated Testing of Android Apps. In Information Technology - New Generations, Shahram Latifi (Ed.). Springer International Publishing, Cham, 571–578.

[2]

David Adamo, Dmitry Nurmuradov, Shraddha Piparia, and Renée Bryce. 2018. Combinatorial-based event sequence testing of Android applications. Information and Software Technology 99 (2018), 98 – 117.

Digital Library

[3]

Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Salvatore De Carmine, and Atif M Memon. 2012. Using GUI ripping for automated testing of Android applications. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE 2012). ACM, New York, NY, USA, 258–261.

Digital Library

[4]

A. Arcuri and L. Briand. 2011. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In 2011 33rd International Conference on Software Engineering (ICSE). 1–10.

Digital Library

[5]

Gigon Bae, Gregg Rothermel, and Doo-Hwan Bae. 2014. Comparing model-based and dynamic event-extraction based GUI testing techniques: An empirical study. Journal of Systems and Software 97, 15–46.

Digital Library

[6]

Young-Min Baek and Doo-Hwan Bae. 2016. Automated Model-based Android GUI Testing Using Multi-level GUI Comparison Criteria. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016). ACM, New York, NY, USA, 238–249.

Digital Library

[7]

Sebastian Bauersfeld and Tanja EJ Vos. 2014. User interface level testing with TESTAR; what about more sophisticated action specification and selection?. In CEUR Workshop Proceedings, Vol. 1354. 60–78.

[8]

Santo Carino. 2016. Dynamically Testing Graphical User Interfaces. Ph.D. Dissertation. The University of Western Ontario, London, ON, CA.

[9]

Wontae Choi, George Necula, and Koushik Sen. 2013. Guided GUI Testing of Android Apps with Minimal Restart and Approximate Learning. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages and Applications (OOPSLA ’13). ACM, New York, NY, USA, 623–640.

Digital Library

[10]

Ana Rosario Espada, María del Mar Gallardo, Alberto Salmerón, and Pedro Merino. 2015. Using Model Checking to Generate Test Cases for Android Applications. In Tenth Workshop on Model-Based Testing (MBT 2015). 7–21.

[11]

JS Foundation. 2018. Appium: Mobile App Automation Made Awesome. http: //appium.io/. Accessed: 2018-07-02.

[12]

Tianxiao Gu, Chun Cao, Tianchi Liu, Chengnian Sun, Jing Deng, Xiaoxing Ma, and Jian Lu. 2017. AimDroid: Activity-Insulated Multi-level Automated Testing for Android Applications. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). 103–114.

[13]

L. V. Haoyin. 2017. Automatic android application GUI testing – A random walk approach. In International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET). 72–76.

[14]

J. Harty, M. Katara, and T. Takala. 2011. Experiences of System-Level Model-Based GUI Testing of an Android Application. In 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation(ICST). 377–386.

Digital Library

[15]

Cuixiong Hu and Iulian Neamtiu. 2011. Automating GUI Testing for Android Applications. In Proceedings of the 6th International Workshop on Automation of Software Test (AST 2011). ACM, New York, NY, USA, 77–83.

Digital Library

[16]

IDC Research. 2017. Smartphone OS Market Share, 2017 Q1. https://www.idc. com/promo/smartphone-market-share/os. Accessed: 2018-02-21.

[17]

Pavneet Singh Kochhar, Thung Ferdian, Nachiappan Nagappan, Thomas Zimmermann, and David Lo. 2015. Understanding the Test Automation Culture of App Developers. In IEEE 8th International Conference on Software Testing, Verification and Validation (ICST). IEEE, 1–10.

[18]

F-Droid Limited and Contributors. 2018. F-Droid - Free and Open Source Android App Repository. https://f-droid.org/. Accessed: 2018-02-10.

[19]

Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An Input Generation System for Android Apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). ACM, New York, NY, USA, 224–234.

Digital Library

[20]

Leonardo Mariani, Mauro Pezzé, Oliviero Riganelli, and Mauro Santoro. 2014. Automatic Testing of GUI-based Applications. Software Testing, Verification and Reliability 24, 5 (2014), 341–366.

Digital Library

[21]

Android Open Source Project. 2018. Android UI/Application Exerciser Monkey. https://developer.android.com/studio/test/monkey.html. Accessed: 2018-07-02.

[22]

Android Open Source Project. 2018. UIAutomator. https://developer.android. com/training/testing/ui-testing/uiautomator-testing.html. Accessed: 2018-07-02.

[23]

Vlad Roubtsov. 2005. Emma. http://emma.sourceforge.net/. Accessed: 2018-03-12.

[24]

Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, Stochastic Model-based GUI Testing of Android Apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2017). ACM, New York, NY, USA, 245–256.

Digital Library

[25]

Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction. MIT press.

Digital Library

[26]

Martijn van Otterlo and Marco Wiering. 2012. Reinforcement Learning and Markov Decision Processes. In Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn van Otterlo (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 3–42.

[27]

Hui Ye, Shaoyin Cheng, Lanbo Zhang, and Fan Jiang. 2013. DroidFuzzer: Fuzzing the Android Apps with Intent-Filter Tag. In Proceedings of International Conference on Advances in Mobile Computing and Multimedia (MoMM ’13). ACM, New York, NY, USA, 68–74.

Digital Library

Cited By

Alenzi A(2024)Element and Event-Based Test Suite Reduction for Android Test Suites Generated by Reinforcement Learningundefined10.12794/metadc2356236Online publication date: Jul-2024
https://doi.org/10.12794/metadc2356236
Yu SFang CLi XLing YChen ZSu Z(2024)Effective, Platform-Independent GUI Testing via Image Embedding and Reinforcement LearningACM Transactions on Software Engineering and Methodology10.1145/367472833:7(1-27)Online publication date: 21-Jun-2024
https://dl.acm.org/doi/10.1145/3674728
Zhao YHarrison BYu T(2024)DinoDroid: Testing Android Apps Using Deep Q-NetworksACM Transactions on Software Engineering and Methodology10.1145/365215033:5(1-24)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3652150
Show More Cited By

Recommendations

Automated model-based Android GUI testing using multi-level GUI comparison criteria
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Automated Graphical User Interface (GUI) testing is one of the most widely used techniques to detect faults in mobile applications (apps) and to test functionality and usability. GUI testing exercises behaviors of an application under test (AUT) by ...
Deep Reinforcement Learning based Android Application GUI Testing
SBES '21: Proceedings of the XXXV Brazilian Symposium on Software Engineering

The advances in mobile computing and the market demand for new products which meet an increasingly public represent the importance to assure the quality of mobile applications. In this context, automated GUI testing has become highlighted in research. ...
Guided, stochastic model-based GUI testing of Android apps
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

Mobile apps are ubiquitous, operate in complex environments and are developed under the time-to-market pressure. Ensuring their correctness and reliability thus becomes an important challenge. This paper introduces Stoat, a novel guided approach to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

A-TEST 2018: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

November 2018

66 pages

ISBN:9781450360531

DOI:10.1145/3278186

General Chair:
Wishnu Prasetya,
Program Chairs:
Tanja E. J. Vos,
Sinem Getir

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '18

Sponsor:

SIGSOFT

ESEC/FSE '18: 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 5, 2018

FL, Lake Buena Vista, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
964
Total Downloads

Downloads (Last 12 months)67
Downloads (Last 6 weeks)8

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Alenzi A(2024)Element and Event-Based Test Suite Reduction for Android Test Suites Generated by Reinforcement Learningundefined10.12794/metadc2356236Online publication date: Jul-2024
https://doi.org/10.12794/metadc2356236
Yu SFang CLi XLing YChen ZSu Z(2024)Effective, Platform-Independent GUI Testing via Image Embedding and Reinforcement LearningACM Transactions on Software Engineering and Methodology10.1145/367472833:7(1-27)Online publication date: 21-Jun-2024
https://dl.acm.org/doi/10.1145/3674728
Zhao YHarrison BYu T(2024)DinoDroid: Testing Android Apps Using Deep Q-NetworksACM Transactions on Software Engineering and Methodology10.1145/365215033:5(1-24)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3652150
Yu SFang CDu MDing ZChen ZSu Z(2024)Practical, Automated Scenario-Based Mobile App TestingIEEE Transactions on Software Engineering10.1109/TSE.2024.341467250:7(1949-1966)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3414672
Gong CYang ZBai YHe JShi JLi KSinha AXu BHou XLo DWang T(2024)Baffle: Hiding Backdoors in Offline Reinforcement Learning Datasets2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00224(2086-2104)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00224
Nayab SWotawa F(2024)Testing and Reinforcement Learning - A Structured Literature Review2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00049(326-335)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00049
Li YLi YYang Y(2024)Test-Agent: A Multimodal App Automation Testing Framework Based on the Large Language Model2024 IEEE 4th International Conference on Digital Twins and Parallel Intelligence (DTPI)10.1109/DTPI61353.2024.10778901(609-614)Online publication date: 18-Oct-2024
https://doi.org/10.1109/DTPI61353.2024.10778901
Abo-eleneen APalliyali ACatal C(2024)The role of Reinforcement Learning in software testingInformation and Software Technology10.1016/j.infsof.2023.107325164:COnline publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107325
Wang FTao CGao J(2024)REDQT: a method for automated mobile application GUI testing based on deep reinforcement learning algorithmsService Oriented Computing and Applications10.1007/s11761-024-00413-yOnline publication date: 1-Jul-2024
https://doi.org/10.1007/s11761-024-00413-y
Tao CWang FGao YGuo HGao J(2024)A reinforcement learning-based approach to testing GUI of moblie applicationsWorld Wide Web10.1007/s11280-024-01252-927:2Online publication date: 22-Feb-2024
https://doi.org/10.1007/s11280-024-01252-9
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten