Yifei Lu
Minxue Pan
ABSTRACT
Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore, need to be thoroughly tested. Unfortunately, the specific preferences used in test cases are typically not explicitly specified, forcing testers to manually set options or blindly try different option combinations. To effectively test the impacts of different preference options, this paper presents PREFEST, as a preference-wise enhanced automatic testing approach, for Android apps. Given a set of test cases, PREFEST can locate the preferences that may affect the test cases with a static and dynamic combined analysis on the app under test, and execute these test cases only under necessary option combinations. The evaluation shows that PREFEST can improve 6.8% code coverage and 12.3% branch coverage and find five more real bugs compared to testing with the original test cases. The test cost is reduced by 99% for both the number of test cases and the testing time, compared to testing under pairwise combination of options.
- Bestoun S Ahmed and Kamal Z Zamli. 2010. PSTG: A T-Way Strategy Adopting Particle Swarm Optimization. In Proceedings of the 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation. IEEE Computer Society, 1–5. Google Scholar
Digital Library
- Domenico Amalfitano, Nicola Amatucci, Anna Rita Fasolino, Porfirio Tramontana, Emily Kowalczyk, and Atif M Memon. 2015. Exploiting the saturation effect in automatic random testing of Android applications. In Proceedings of the Second ACM International Conference on Mobile Software Engineering and Systems. IEEE Press, 33–43. Google ScholarDigital Library
- Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Salvatore De Carmine, and Atif M Memon. 2012. Using GUI ripping for automated testing of Android applications. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ACM, 258–261. Google ScholarDigital Library
- Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. ACM, 59. Google ScholarDigital Library
- AppiumConf. 2019. Appium : Automation for Apps. http://appium.io/. {online, accessed 15-Feb-2019}.Google Scholar
- Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. 2014.Google ScholarDigital Library
- Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps. Acm Sigplan Notices 49, 6 (2014), 259–269. Google ScholarDigital Library
- Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of Android apps. In Acm Sigplan Notices, Vol. 48. ACM, 641–660. Google ScholarDigital Library
- Wontae Choi, George Necula, and Koushik Sen. 2013. Guided GUI testing of Android apps with minimal restart and approximate learning. In Acm Sigplan Notices, Vol. 48. ACM, 623–640. Google ScholarDigital Library
- Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. 2015. Automated Test Input Generation for Android: Are We There Yet?(E). (2015), 429–440.Google Scholar
- David M Cohen, Siddhartha R Dalal, Michael L Fredman, and Gardner C Patton. 1997. The AETG System: An Approach to Testing Based on Combinatorial Design. IEEE Transactions on Software Engineering 23, 7 (1997), 437–444. Google ScholarDigital Library
- Myra B Cohen, Matthew B Dwyer, and Jiangfan Shi. 2008. Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach. IEEE Transactions on Software Engineering 34, 5 (2008), 633–650. Google ScholarDigital Library
- Google Developers. 2018. Documentation of Settings for Android Developers. https://developer.android.com/guide/topics/ui/settings. {online, accessed 01-Sep- 2018}.Google Scholar
- Google Developers. 2019. UI/Application Exerciser Monkey. https://developer. android.com/studio/test/monkey. {online, accessed 15-Feb-2019}.Google Scholar
- EclEmmaTeam. 2019. JaCoCo: Java Code Coverage Library. https://www. eclemma.org/jacoco/. {online, accessed 15-Feb-2019}.Google Scholar
- Shuai Hao, Bin Liu, Suman Nath, William GJ Halfond, and Ramesh Govindan. 2014. PUMA: programmable UI-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 204–217. Google ScholarDigital Library
- Casper S Jensen, Mukul R Prasad, and Anders Møller. 2013. Automated testing with targeted event sequence generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis. ACM, 67–77. Google ScholarDigital Library
- Patrick Lam, Eric Bodden, Ondrej Lhoták, and Laurie Hendren. 2011. The Soot framework for Java program analysis: a retrospective. In Cetus Users and Compiler Infastructure Workshop (CETUS 2011), Vol. 15. 35.Google Scholar
- Yu Lei, Raghu Kacker, D Richard Kuhn, Vadim Okun, and James Lawrence. 2007. IPOG: A General Strategy for T-Way Software Testing. In Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems. IEEE Computer Society, 549–556. Google ScholarDigital Library
- Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for Android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 224–234. Google ScholarDigital Library
- Riyadh Mahmood, Nariman Mirzaei, and Sam Malek. 2014. Evodroid: Segmented evolutionary testing of Android apps. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 599–609. Google ScholarDigital Library
- Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, 94–105. Google ScholarDigital Library
- CC Michael, GE McGraw, MA Schatz, and CC Walton. 1997. Genetic algorithms for dynamic test data generation. In Proceedings of the 12th international conference on Automated software engineering (formerly: KBSE). IEEE Computer Society, 307. Google ScholarDigital Library
- Nariman Mirzaei, Hamid Bagheri, Riyadh Mahmood, and Sam Malek. 2015. SIGDroid: Automated system input generation for Android applications. In Proceedings of the 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE). IEEE Computer Society, 461–471. Google ScholarDigital Library
- Nariman Mirzaei, Joshua Garcia, Hamid Bagheri, Alireza Sadeghi, and Sam Malek. 2016. Reducing combinatorics in GUI testing of Android applications. In Proceedings of the 38th International Conference on Software Engineering. ACM, 559–570. Google ScholarDigital Library
- monkeyrunner. 2019. monkeyrunner. https://developer.android.com/studio/test/ monkeyrunner/. {online, accessed 15-Feb-2019}.Google Scholar
- Changhai Nie and Hareton Leung. 2011. A survey of combinatorial testing. ACM Computing Surveys (CSUR) 43, 2 (2011), 11. Google ScholarDigital Library
- pcqpcq. 2019. open-source-android-apps. https://github.com/pcqpcq/opensource-android-apps/. {online, accessed 15-Feb-2019}.Google Scholar
- RobotiumTech. 2019. Android UI Testing Robotium. https://github.com/ RobotiumTech/robotium. {online, accessed 15-Feb-2019}.Google Scholar
- Alireza Sadeghi, Reyhaneh Jabbarvand, and Sam Malek. 2017. PATDdroid: permission-aware GUI testing of Android. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 220–232. Google ScholarDigital Library
- Toshiaki Shiba, Tatsuhiro Tsuchiya, and Tohru Kikuno. 2004. Using Artificial Life Techniques to Generate Test Cases for Combinatorial Testing. In Proceedings of the 28th Annual International Computer Software and Applications Conference-Volume 01. IEEE Computer Society, 72–77. Google ScholarDigital Library
- Skylot. 2019. jadx : Dex to Java decompiler. https://github.com/skylot/jadx/. {online, accessed 15-Feb-2019}.Google Scholar
- Wei Song, Xiangxing Qian, and Jeff Huang. 2017. EHBDroid: beyond GUI testing for Android applications. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 27–37. Google ScholarDigital Library
- Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 245–256. Google ScholarDigital Library
Index Terms
- Preference-wise testing for Android applications
Recommendations
Preference-wise Testing of Android Apps via Test Amplification
Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, ...
Sapienz: multi-objective automated testing for Android applications
ISSTA 2016: Proceedings of the 25th International Symposium on Software Testing and AnalysisWe introduce Sapienz, an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising length, while simultaneously maximising coverage and fault revelation. Sapienz combines ...
A compatibility testing platform for android multimedia applications
Along with the widespread use of smartphones, Android has become one of the major platforms for multimedia applications (apps). However, due to the fast evolution of Android operating system and the fragmentation of Android devices, it becomes important ...
Comments