research-article

Minimizing GUI event traces

Authors:

Osbert Bastani,

Alex AikenAuthors Info & Claims

FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

Pages 422 - 434

https://doi.org/10.1145/2950290.2950342

Published: 01 November 2016 Publication History

Abstract

GUI input generation tools for Android apps, such as Android's Monkey, are useful for automatically producing test inputs, but these tests are generally orders of magnitude larger than necessary, making them difficult for humans to understand. We present a technique for minimizing the output of such tools. Our technique accounts for the non-deterministic behavior of mobile apps, producing small event traces that reach a desired activity with high probability.

We propose a variant of delta debugging, augmented to handle non-determinism, to solve the problem of trace minimization. We evaluate our algorithm on two sets of commercial and open-source Android applications, showing that we can minimize large event traces reaching a particular application activity, producing traces that are, on average, less than 2% the size of the original traces.

References

[1]

D. Amalfitano, A. R. Fasolino, P. Tramontana, S. D. Carmine, and A. M. Memon. Using GUI ripping for automated testing of Android applications. In IEEE/ACM International Conference on Automated Software Engineering, ASE’12, Essen, Germany, September 3-7, 2012, pages 258–261, 2012.

Digital Library

[2]

D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon. Mobiguitar: Automated modelbased testing of mobile apps. IEEE Software, 32(5):53– 59, 2015.

Digital Library

[3]

S. Anand, M. Naik, M. J. Harrold, and H. Yang. Automated concolic testing of smartphone apps. In 20th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-20), SIGSOFT/FSE’12, Cary, NC, USA - November 11 - 16, 2012, page 59, 2012.

Digital Library

[4]

T. Azim and I. Neamtiu. Targeted and depth-first exploration for systematic testing of Android apps. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, part of SPLASH 2013, Indianapolis, IN, USA, October 26-31, 2013, pages 641–660, 2013.

Digital Library

[5]

R. Bhoraskar, S. Han, J. Jeon, T. Azim, S. Chen, J. Jung, S. Nath, R. Wang, and D. Wetherall. Brahmastra: Driving apps to test the security of third-party components. In Proceedings of the 23rd USENIX Security Symposium, San Diego, CA, USA, August 20-22, 2014., pages 1021–1036, 2014.

Digital Library

[6]

M. Burger and A. Zeller. Minimizing reproduction of software failures. In Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17-21, 2011, pages 221–231, 2011.

Digital Library

[7]

T. L. Cheung, K. Okamoto, F. M. III, X. Liu, and V. Akella. Markov decision process (MDP) framework for optimizing software on mobile phones. In Proceedings of the 9th ACM & IEEE International conference on Embedded software, EMSOFT 2009, Grenoble, France, October 12-16, 2009, pages 11–20, 2009.

Digital Library

[8]

J. Choi and A. Zeller. Isolating failure-inducing thread schedules. In Proceedings of the 11th International Symposium on Software Testing and Analysis, ISSTA 2002, Rome, Italy, July 22-24, 2002, pages 210–220, 2002.

Digital Library

[9]

W. Choi, G. C. Necula, and K. Sen. Guided GUI testing of Android apps with minimal restart and approximate learning. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013, part of SPLASH 2013, Indianapolis, IN, USA, October 26-31, 2013, pages 623–640, 2013.

Digital Library

[10]

S. R. Choudhary, A. Gorla, and A. Orso. Automated test input generation for Android: Are we there yet? In 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pages 429–440, 2015.

Digital Library

[11]

L. Gomez, I. Neamtiu, T. Azim, and T. D. Millstein. RERAN: timing- and touch-sensitive record and replay for Android. In 35th International Conference on Software Engineering, ICSE ’13, San Francisco, CA, USA, May 18-26, 2013, pages 72–81, 2013.

Digital Library

[12]

Google. Espresso - https://developer.android.com/training/testing/uitesting/espresso-testing.html.

[13]

Google. UI/Application exerciser monkey - https://developer.android.com/tools/help/monkey.html.

[14]

M. Halpern, Y. Zhu, R. Peri, and V. J. Reddi. Mosaic: cross-platform user-interaction record and replay for the fragmented Android ecosystem. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS 2015, Philadelphia, PA, USA, March 29-31, 2015, pages 215–224, 2015.

[15]

S. Hao, B. Liu, S. Nath, W. G. J. Halfond, and R. Govindan. PUMA: programmable ui-automation for large-scale dynamic analysis of mobile apps. In The 12th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys’14, Bretton Woods, NH, USA, June 16-19, 2014, pages 204–217, 2014.

Digital Library

[16]

Y. Hu, T. Azim, and I. Neamtiu. Versatile yet lightweight record-and-replay for Android. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2015, part of SLASH 2015, Pittsburgh, PA, USA, October 25-30, 2015, pages 349–366, 2015.

Digital Library

[17]

L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. J. Artif. Intell. Res. (JAIR), 4:237–285, 1996.

Digital Library

[18]

K. Lee, J. Flinn, T. J. Giuli, B. Noble, and C. Peplin. AMC: verifying user interface properties for vehicular applications. In The 11th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys’13, Taipei, Taiwan, June 25-28, 2013, pages 1–12, 2013.

Digital Library

[19]

B. Liu, S. Nath, R. Govindan, and J. Liu. DECAF: detecting and characterizing ad fraud in mobile apps. In Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2014, Seattle, WA, USA, April 2-4, 2014, pages 57–70, 2014.

Digital Library

[20]

A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: an input generation system for Android apps. In Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE’13, Saint Petersburg, Russian Federation, August 18-26, 2013, pages 224–234, 2013.

Digital Library

[21]

R. Mahmood, N. Mirzaei, and S. Malek. Evodroid: segmented evolutionary testing of Android apps. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16 - 22, 2014, pages 599–609, 2014.

Digital Library

[22]

G. Misherghi and Z. Su. HDD: hierarchical delta debugging. In 28th International Conference on Software Engineering (ICSE 2006), Shanghai, China, May 20- 28, 2006, pages 142–151, 2006.

Digital Library

[23]

S. Nath, F. X. Lin, L. Ravindranath, and J. Padhye. Smartads: bringing contextual ads to mobile apps. In The 11th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys’13, Taipei, Taiwan, June 25-28, 2013, pages 111–124, 2013.

Digital Library

[24]

V. Rastogi, Y. Chen, and W. Enck. Appsplayground: automatic security analysis of smartphone applications. In Third ACM Conference on Data and Application Security and Privacy, CODASPY’13, San Antonio, TX, USA, February 18-20, 2013, pages 209–220, 2013.

Digital Library

[25]

L. Ravindranath, S. Nath, J. Padhye, and H. Balakrishnan. Automatic and scalable fault detection for mobile applications. In The 12th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys’14, Bretton Woods, NH, USA, June 16-19, 2014, pages 190–203, 2014.

Digital Library

[26]

Robotium. Robotium - https://github.com/robotiumtech/robotium.

[27]

R. Sasnauskas and J. Regehr. Intent fuzzer: crafting intents of death. In Proceedings of the 2014 Joint International Workshop on Dynamic Analysis (WODA) and Software and System Performance Testing, Debugging, and Analytics (PERTEA), WODA+PERTEA 2014, San Jose, CA, USA, July 22, 2014, pages 1–5, 2014.

Digital Library

[28]

C. Scott, A. Panda, V. Brajkovic, G. Necula, A. Krishnamurthy, and S. Shenker. Minimizing faulty executions of distributed systems. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16), pages 291–309, Santa Clara, CA, 2016. USENIX Association.

Digital Library

[29]

C. Scott, A. Wundsam, B. Raghavan, A. Panda, A. Or, J. Lai, E. Huang, Z. Liu, A. El-Hassany, S. Whitlock, H. B. Acharya, K. Zarifis, and S. Shenker. Troubleshooting blackbox SDN control software with minimal causal sequences. In ACM SIGCOMM 2014 Conference, SIGCOMM’14, Chicago, IL, USA, August 17- 22, 2014, pages 395–406, 2014.

Digital Library

[30]

Selendroid. Selendroid - http://selendroid.io/.

[31]

R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction, volume 28. MIT press, 1998.

Digital Library

[32]

F. Wilcoxon. Individual comparisons by ranking methods. Biometrics bulletin, 1(6):80–83, 1945.

[33]

Xamarin. Calabash - http://calaba.sh/.

[34]

W. Yang, M. R. Prasad, and T. Xie. A grey-box approach for automated gui-model generation of mobile applications. In Fundamental Approaches to Software Engineering - 16th International Conference, FASE 2013, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2013, Rome, Italy, March 16-24, 2013. Proceedings, pages 250–265, 2013.

Digital Library

[35]

H. Ye, S. Cheng, L. Zhang, and F. Jiang. Droidfuzzer: Fuzzing the Android apps with intent-filter tag. In The 11th International Conference on Advances in Mobile Computing & Multimedia, MoMM ’13, Vienna, Austria, December 2-4, 2013, page 68, 2013.

Digital Library

[36]

A. Zeller. Yesterday, my program worked. today, it does not. why? In Software Engineering - ESEC/FSE’99, 7th European Software Engineering Conference, Held Jointly with the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, Toulouse, France, September 1999, pages 253–267, 1999.

Digital Library

[37]

A. Zeller. Isolating cause-effect chains from computer programs. In Proceedings of the Tenth ACM SIGSOFT Symposium on Foundations of Software Engineering 2002, Charleston, South Carolina, USA, November 18- 22, 2002, pages 1–10, 2002.

Digital Library

[38]

A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Trans. Software Eng., 28(2):183–200, 2002.

Digital Library

Cited By

Jiang MTang CZhang XZhao YDing Z(2024)DDImage: an image reduction based approach for automatically explaining black-box classifiersEmpirical Software Engineering10.1007/s10664-024-10505-029:5Online publication date: 30-Jul-2024
https://doi.org/10.1007/s10664-024-10505-0
Vince DKiss Á(2024)Evaluation of the fixed‐point iteration of minimizing delta debuggingJournal of Software: Evolution and Process10.1002/smr.2702Online publication date: 23-Jun-2024
https://doi.org/10.1002/smr.2702
Jiang MTang CZhang XZhao YDing Z(2023)Automated Image Reduction for Explaining Black-box Classifiers2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00042(367-378)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00042
Show More Cited By

Index Terms

Minimizing GUI event traces
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Analyzing GUI running fluency for Android apps
MSCC '16: Proceedings of the 3rd ACM Workshop on Mobile Sensing, Computing and Communication

Android as a free open platform has become increasingly popular and been widespread adopted in mobile, tablet, and other devices. However, a great number of issues, such as inadequate quality and the fragmentation phenomenon, have emerged, enhancing the ...
Testing android apps through symbolic execution

There is a growing need for automated testing techniques aimed at Android apps. A critical challenge is the systematic generation of test cases. One method of systematically generating test cases for Java programs is symbolic execution. But applying ...
SimplyDroid: efficient event sequence simplification for Android application
ASE '17: Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering

To ensure the quality of Android applications, many automatic test case generation techniques have been proposed. Among them, the Monkey fuzz testing tool and its variants are simple, effective and widely applicable. However, one major drawback of the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

November 2016

1156 pages

ISBN:9781450342186

DOI:10.1145/2950290

General Chair:
Thomas Zimmermann
Microsoft Research, USA
,
Program Chairs:
Jane Cleland-Huang
University of Notre Dame, USA
,
Zhendong Su
University of California at Davis, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

FSE'16

Sponsor:

SIGSOFT

FSE'16: 24nd ACM SIGSOFT International Symposium on the Foundations of Software Engineering

November 13 - 18, 2016

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
380
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang MTang CZhang XZhao YDing Z(2024)DDImage: an image reduction based approach for automatically explaining black-box classifiersEmpirical Software Engineering10.1007/s10664-024-10505-029:5Online publication date: 30-Jul-2024
https://doi.org/10.1007/s10664-024-10505-0
Vince DKiss Á(2024)Evaluation of the fixed‐point iteration of minimizing delta debuggingJournal of Software: Evolution and Process10.1002/smr.2702Online publication date: 23-Jun-2024
https://doi.org/10.1002/smr.2702
Jiang MTang CZhang XZhao YDing Z(2023)Automated Image Reduction for Explaining Black-box Classifiers2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00042(367-378)Online publication date: Mar-2023
https://doi.org/10.1109/SANER56733.2023.00042
Ran DWang HWang WXie T(2023)Badge: Prioritizing UI Events with Hierarchical Multi-Armed Bandits for Automated UI Testing2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00083(894-905)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00083
Nie LSaid KMa LZheng YZhao Y(2023)A systematic mapping study for graphical user interface testing on mobile appsIET Software10.1049/sfw2.1212317:3(249-267)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1049/sfw2.12123
Bársony DKiss ÁMarín BSaadatmand M(2022)OpenGL API call trace reduction with the minimizing Delta debugging algorithmProceedings of the 13th International Workshop on Automating Test Case Design, Selection and Evaluation10.1145/3548659.3561308(53-56)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3548659.3561308
Su TFan LChen SLiu YXu LPu GSu Z(2022)Why My App Crashes? Understanding and Benchmarking Framework-Specific Exceptions of Android AppsIEEE Transactions on Software Engineering10.1109/TSE.2020.301343848:4(1115-1137)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TSE.2020.3013438
Zhong YShi MXu YFang CChen Z(2022)Iterative Android automated testingFrontiers of Computer Science10.1007/s11704-022-1658-817:5Online publication date: 24-Dec-2022
https://doi.org/10.1007/s11704-022-1658-8
Vince DHodován RBársony DKiss Á(2022)The effect of hoisting on variants of Hierarchical Delta DebuggingJournal of Software: Evolution and Process10.1002/smr.248334:11Online publication date: 19-Jun-2022
https://doi.org/10.1002/smr.2483
Yasin HHamid SRaja Yusof R(2021)DroidbotX: Test Case Generation Tool for Android Applications Using Q-LearningSymmetry10.3390/sym1302031013:2(310)Online publication date: 12-Feb-2021
https://doi.org/10.3390/sym13020310
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten