Tongtong Xu
Minxue Pan
ABSTRACT
GUI testing is an essential part of regression testing for Android apps. For regression GUI testing to remain effective, it is important that obsolete GUI test scripts get repaired after the app has evolved. In this paper, we propose a novel approach named GUIDER to automated repair of GUI test scripts for Android apps. The key novelty of the approach lies in the utilization of both structural and visual information of widgets on app GUIs to better understand what widgets of the base version app become in the updated version. A supporting tool has been implemented for the approach. Experiments conducted on the popular messaging and social media app WeChat show that GUIDER is both effective and efficient. Repairs produced by GUIDER enabled 88.8% and 54.9% more test actions to run correctly than those produced by existing approaches to GUI test repair that rely solely on visual or structural information of app GUIs.
- Gigon Bae, Gregg Rothermel, and Doo-Hwan Bae. 2014. Comparing Model-Based and Dynamic Event-Extraction Based GUI Testing Techniques. J. Syst. Softw., 97, C (2014), Oct., 15–46. issn:0164-1212 https://doi.org/10.1016/j.jss.2014.06.039 Google Scholar
Digital Library
- Nana Chang, Linzhang Wang, Yu Pei, Subrota K. Mondal, and Xuandong Li. 2018. Change-Based Test Script Maintenance for Android Apps. In 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS). 215–225. https://doi.org/10.1109/QRS.2018.00035 Google ScholarCross Ref
- Shauvik Roy Choudhary, Dan Zhao, Husayn Versee, and Alessandro Orso. 2011. WATER: Web Application TEst Repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering (ETSE ’11). Association for Computing Machinery, New York, NY, USA. 24–29. isbn:9781450308083 https://doi.org/10.1145/2002931.2002935 Google ScholarDigital Library
- Brett Daniel, Tihomir Gvero, and Darko Marinov. 2010. On Test Repair Using Symbolic Execution. In Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA ’10). Association for Computing Machinery, New York, NY, USA. 207–218. isbn:9781605588230 https://doi.org/10.1145/1831708.1831734 Google ScholarDigital Library
- Brett Daniel, Vilas Jagannath, Danny Dig, and Darko Marinov. 2009. ReAssert: Suggesting Repairs for Broken Unit Tests. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (ASE ’09). IEEE Computer Society, USA. 433–444. isbn:9780769538914 https://doi.org/10.1109/ASE.2009.17 Google ScholarDigital Library
- Brett Daniel, Qingzhou Luo, Mehdi Mirzaaghaei, Danny Dig, Darko Marinov, and Mauro Pezzè. 2011. Automated GUI Refactoring and Test Script Repair. In Proceedings of the First International Workshop on End-to-End Test Script Engineering (ETSE ’11). Association for Computing Machinery, New York, NY, USA. 38–41. isbn:9781450308083 https://doi.org/10.1145/2002931.2002937 Google ScholarDigital Library
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’08/ETAPS’08). Springer-Verlag, Berlin, Heidelberg. 337–340. isbn:3540787992 https://doi.org/10.1007/978-3-540-78800-3_24 Google ScholarCross Ref
- Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. Amsterdam, Netherlands.Google Scholar
- Chen Fu, Mark Grechanik, and Qing Xie. 2009. Inferring Types of References to GUI Objects in Test Scripts. In Proceedings of the 2009 International Conference on Software Testing Verification and Validation (ICST ’09). IEEE Computer Society, USA. 1–10. isbn:9780769536019 https://doi.org/10.1109/ICST.2009.12 Google ScholarDigital Library
- Zebao Gao, Zhenyu Chen, Yunxiao Zou, and Atif M. Memon. 2016. SITAR: GUI Test Script Repair. IEEE Trans. Softw. Eng., 42, 2 (2016), Feb., 170–186. issn:0098-5589 https://doi.org/10.1109/TSE.2015.2454510 Google ScholarDigital Library
- Mark Grechanik, Qing Xie, and Chen Fu. 2009. Maintaining and Evolving GUI-Directed Test Scripts. In Proceedings of the 31st International Conference on Software Engineering (ICSE ’09). Association for Computing Machinery, New York, NY, USA. 408–418. isbn:9781424434534 https://doi.org/10.1109/ICSE.2009.5070540 Google ScholarDigital Library
- Mouna Hammoudi, Gregg Rothermel, and Andrea Stocco. 2016. WATERFALL: An Incremental Approach for Repairing Record-Replay Tests of Web Applications. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2016). Association for Computing Machinery, New York, NY, USA. 751–762. isbn:9781450342186 https://doi.org/10.1145/2950290.2950294 Google ScholarDigital Library
- Si Huang, Myra B. Cohen, and Atif M. Memon. 2010. Repairing GUI Test Suites Using a Genetic Algorithm. In Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation (ICST ’10). IEEE Computer Society, USA. 245–254. isbn:9780769539904 https://doi.org/10.1109/ICST.2010.39 Google ScholarDigital Library
- JS Foundation. 2020. Appium: Mobile App Automation Made Awesome. http://appium.io/. [Online; accessed 20-August-2020].Google Scholar
- Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Cristiano Spadaro. 2013. Comparing the Maintainability of Selenium WebDriver Test Suites Employing Different Locators: A Case Study. In Proceedings of the 2013 International Workshop on Joining AcadeMiA and Industry Contributions to Testing Automation (JAMAICA 2013). Association for Computing Machinery, New York, NY, USA. 53–58. isbn:9781450321617 https://doi.org/10.1145/2489280.2489284 Google ScholarDigital Library
- Xiao Li, Nana Chang, Yan Wang, Haohua Huang, Yu Pei, Linzhang Wang, and Xuandong Li. 2017. ATOM: Automatic Maintenance of GUI Test Scripts for Evolving Mobile Applications. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST). 161–171. https://doi.org/10.1109/ICST.2017.22 Google ScholarCross Ref
- David G. Lowe. 1999. Object recognition from local scale-invariant features. In Proceedings of the Seventh IEEE International Conference on Computer Vision. 2, 1150–1157 vol.2. https://doi.org/10.1109/ICCV.1999.790410 Google ScholarCross Ref
- David G. Lowe. 2004. Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vision, 60, 2 (2004), Nov., 91–110. issn:0920-5691 https://doi.org/10.1023/B:VISI.0000029664.99615.94 Google ScholarDigital Library
- Atif M. Memon. 2007. An Event-Flow Model of GUI-Based Applications for Testing: Research Articles. Softw. Test. Verif. Reliab., 17, 3 (2007), Sept., 137–157. issn:0960-0833 https://doi.org/10.1002/stvr.364 Google ScholarCross Ref
- Atif M. Memon and Mary Lou Soffa. 2003. Regression Testing of GUIs. SIGSOFT Softw. Eng. Notes, 28, 5, 118–127. issn:0163-5948 https://doi.org/10.1145/949952.940088 Google ScholarDigital Library
- OpenCV team. 2020. OpenCV library. https://opencv.org/. [Online; accessed 20-August-2020].Google Scholar
- Minxue Pan, Tongtong Xu, Yu Pei, Zhong Li, Tian Zhang, and Xuandong Li. 2020. GUI-Guided Test Script Repair for Mobile Apps. IEEE Transactions on Software Engineering, 1–1. https://doi.org/10.1109/TSE.2020.3007664 Google ScholarCross Ref
- Shikha Raina and Arun Prakash Agarwal. 2013. An Automated Tool for Regression Testing in Web Applications. SIGSOFT Softw. Eng. Notes, 38, 4 (2013), July, 1–4. issn:0163-5948 https://doi.org/10.1145/2492248.2492272 Google ScholarDigital Library
- Robotium.org. 2020. Android UI Testing. http://www.robotium.org. [Online; accessed 20-August-2020].Google Scholar
- Statista. 2020. Number of monthly active WeChat users from 2nd quarter 2011 to 3rd quarter 2020. https://www.statista.com/statistics/255778/number-of-active-wechat-messenger-accounts/. [Online; accessed 12-December-2020].Google Scholar
- Andrea Stocco, Rahulkrishna Yandrapally, and Ali Mesbah. 2018. Visual Web Test Repair. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018). Association for Computing Machinery, New York, NY, USA. 503–514. isbn:9781450355735 https://doi.org/10.1145/3236024.3236063 Google ScholarDigital Library
- Guowei Yang, Sarfraz Khurshid, and Miryung Kim. 2012. Specification-Based Test Repair Using a Lightweight Formal Method. In FM 2012: Formal Methods, Dimitra Giannakopoulou and Dominique Méry (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 455–470. isbn:978-3-642-32759-9 https://doi.org/10.1007/978-3-642-32759-9_37 Google ScholarCross Ref
- Sai Zhang, Hao Lü, and Michael D. Ernst. 2013. Automatically Repairing Broken Workflows for Evolving GUI Applications. In Proceedings of the 2013 International Symposium on Software Testing and Analysis (ISSTA 2013). Association for Computing Machinery, New York, NY, USA. 45–55. isbn:9781450321594 https://doi.org/10.1145/2483760.2483775 Google ScholarDigital Library
Index Terms
- GUIDER: GUI structure and vision co-guided test script repair for Android apps
Recommendations
Guided, stochastic model-based GUI testing of Android apps
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software EngineeringMobile apps are ubiquitous, operate in complex environments and are developed under the time-to-market pressure. Ensuring their correctness and reliability thus becomes an important challenge. This paper introduces Stoat, a novel guided approach to ...
GUI-guided repair of mobile test scripts
ICSE '19: Proceedings of the 41st International Conference on Software Engineering: Companion ProceedingsGraphical User Interface (GUI) testing has been the focus of mobile app testing. Manual test cases, containing valuable human knowledge about the apps under test, are often coded as scripts to enable automated and repeated execution for test cost ...
Preference-wise Testing of Android Apps via Test Amplification
Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, ...
Comments