Skip to main content
SpringerLink
Log in

AppSPIN: reconfiguration-based responsiveness testing and diagnosing for Android Apps

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

App responsiveness is the most intuitive interpretation of App performance from the users’ perspective. Traditional performance profilers only focus on one type of program activity (e.g., CPU profiling). In contrast, the cause of slow responsiveness can be diverse or even due to the joint effect of multiple kinds. Also, various test configurations, such as device hardware and wireless connectivity, can dramatically impact particular program activities and indirectly affect App responsiveness. Conventional mobile testing lacks mechanisms to reveal configuration-sensitive bugs. In this paper, we propose AppSPIN, a tool to diagnose App responsiveness bugs and systematically explore configuration-sensitive bugs automatically. AppSPIN instruments the App to collect program events and UI responsiveness. The instrumented App is exercised with automated monkey testers, and AppSPIN correlates excessive and lengthy program events with poor responsiveness detected at runtime. The diagnosis process also synthesizes the major resource bottleneck for the App under test. After one test run, AppSPIN automatically alters the test configuration with most bottlenecked resources to further explore responsiveness bugs that occur only with particular test configurations. Our experiments with 30 real-world Apps show that AppSPIN can detect 123 unique responsiveness bugs and successfully diagnose the cause for 87% cases with an average of 15-minute test time and negligible overhead. Also, with altered test configurations, AppSPIN uncovers a considerable number of new bugs within four extra test runs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. For instance, Google has a record of nearly 24 thousand different device models as of 2019Q1 (https://support.google.com/googleplay/answer/1727131/).

  2. Stack Overflow, https://stackoverflow.com/

References

  • Android Monkey: An Android Tester that Generates Pseudo-random Test Inputs. https://developer.android.com/studio/test/monkey.html/

  • Android Performance Patterns: Why 60fps? https://www.youtube.com/watch?v=CaMTIgxCSqU/

  • Android Profiler: A Profiler that Provides Real-time Information About CPU, Memory, Network, and Battery Resources. https://developer.android.com/studio/profile/android-profiler/

  • AppetizerIO: An Integrated Platform for Mobile Testing. https://github.com/appetizerio/

  • Appium: An Open Source Test Automation Framework for Use with Native, Hybrid and Mobile Web Apps. http://appium.io/

  • Brocanelli, M., Wang, X.: Hang doctor: runtime detection and diagnosis of soft hangs for smartphone apps. In: Proceedings of the Thirteenth EuroSys Conference, EuroSys 2018, pp. 6–1615 (2018). https://doi.org/10.1145/3190508.3190525

  • Chen, Q.A., Luo, H., Rosen, S., Mao, Z.M., Iyer, K., Hui, J., Sontineni, K., Lau, K.: Qoe doctor: diagnosing mobile app qoe with automated ui control and cross-layer analysis. In: Proceedings of the 2014 Conference on Internet Measurement Conference. IMC ’14, pp. 151–164 (2014). https://doi.org/10.1145/2663716.2663726

  • Chen, Y., Yang, Y., Lei, Z., Xia, M., Qi, Z.: Bootstrapping automated testing for restful web services. In: Guerra, E., Stoelinga, M. (eds.) 24th International Conference on Fundamental Approaches to Software Engineering, FASE 2021, vol. 12649, pp. 46–66 (2021). https://doi.org/10.1007/978-3-030-71500-7_3

  • Das, T., Penta, M.D., Malavolta, I.: A quantitative and qualitative investigation of performance-related commits in android apps. In: 2016 IEEE International Conference on Software Maintenance and Evolution, ICSME 2016, pp. 443–447 (2016). https://doi.org/10.1109/ICSME.2016.49

  • Das, T., Penta, M.D., Malavolta, I.: Characterizing the evolution of statically-detectable performance issues of android apps. Empir. Softw. Eng. 25(4), 2748–2808 (2020). https://doi.org/10.1007/s10664-019-09798-3

    Article  Google Scholar 

  • Gao, Y., Luo, Y., Chen, D., Huang, H., Dong, W., Xia, M., Liu, X., Bu, J.: Every pixel counts: fine-grained UI rendering analysis for mobile applications. In: 2017 IEEE Conference on Computer Communications, INFOCOM 2017, pp. 1–9 (2017). https://doi.org/10.1109/INFOCOM.2017.8057023

  • Google Play Supported Devices. https://support.google.com/googleplay/answer/1727131/

  • Gordon, M.S., Hong, D.K., Chen, P.M., Flinn, J., Mahlke, S., Mao, Z.M.: Accelerating mobile applications through flip-flop replication. In: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. MobiSys ’15, pp. 137–150 (2015). https://doi.org/10.1145/2742647.2742649

  • Habchi, S., Blanc, X., Rouvoy, R.: On adopting linters to deal with performance concerns in android apps. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. ASE 2018, pp. 6–16 (2018). https://doi.org/10.1145/3238147.3238197

  • Hrubý, T., Bos, H., Tanenbaum, A.S.: When slower is faster: on heterogeneous multicores for reliable systems. In: 2013 USENIX Annual Technical Conference, pp. 255–266 (2013)

  • Huang, T., Zhang, Z., Zhu, X.: Analyzing GUI running fluency for android apps. In: Proceedings of the 3rd ACM Workshop on Mobile Sensing, Computing and Communication, MSCC@MobiHoc 2016, pp. 16–22 (2016). https://doi.org/10.1145/2940353.2940356

  • Jovic, M., Adamoli, A., Hauswirth, M.: Catch me if you can: performance bug detection in the wild. In: Proceedings of the 2011 ACM International Conference on Object Oriented Programming Systems Languages and Applications. OOPSLA ’11, pp. 155–170 (2011). https://doi.org/10.1145/2048066.2048081

  • Kang, Y., Zhou, Y., Xu, H., Lyu, M.R.: Diagdroid: android performance diagnosis via anatomizing asynchronous executions. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, pp. 410–421 (2016). https://doi.org/10.1145/2950290.2950316

  • Kwon, Y., Lee, S., Yi, H., Kwon, D., Yang, S., Chun, B.-G., Huang, L., Maniatis, P., Naik, M., Paek, Y.: Mantis: automatic performance prediction for smartphone applications. In: Proceedings of the 2013 USENIX Conference on Annual Technical Conference. USENIX ATC’13, pp. 297–308 (2013)

  • Li, W., Jiang, Y., Ma, J., Xu, C.: Automatic performance testing for image displaying in android apps, pp. 367–376 (2021)

  • Liang, C.M., Lane, N.D., Brouwers, N., Zhang, L., Karlsson, B.F., Liu, H., Liu, Y., Tang, J., Shan, X., Chandra, R., Zhao, F.: Caiipa: automated large-scale mobile app testing through contextual fuzzing. In: The 20th Annual International Conference on Mobile Computing and Networking, MobiCom’14, pp. 519–530 (2014). https://doi.org/10.1145/2639108.2639131

  • Lin, Y., Radoi, C., Dig, D.: Retrofitting concurrency for android applications through refactoring. In: Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. FSE 2014, pp. 341–352 (2014). https://doi.org/10.1145/2635868.2635903

  • Liu, Y., Xu, C., Cheung, S.: Characterizing and detecting performance bugs for smartphone applications. In: 36th International Conference on Software Engineering, ICSE ’14, pp. 1013–1024 (2014). https://doi.org/10.1145/2568225.2568229

  • Machiry, A., Tahiliani, R., Naik, M.: Dynodroid: an input generation system for android apps. In: Meyer, B., Baresi, L., Mezini, M. (eds.) Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE’13, pp. 224–234 (2013). https://doi.org/10.1145/2491411.2491450

  • Maxim: An Efficient Android Monkey Tester. https://github.com/zhangzhao4444/Maxim/

  • Mudduluru, R., Ramanathan, M.K.: Efficient flow profiling for detecting performance bugs. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18–20, 2016, pp. 413–424 (2016). https://doi.org/10.1145/2931037.2931066

  • Pradel, M., Schuh, P., Necula, G., Sen, K.: Eventbreak: analyzing the responsiveness of user interfaces through performance-guided test generation. In: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications. OOPSLA ’14, pp. 33–47 (2014). https://doi.org/10.1145/2660193.2660233

  • Prim’s Algorithm. https://en.wikipedia.org/wiki/Prim%27s_algorithm/

  • Ravindranath, L., Padhye, J., Agarwal, S., Mahajan, R., Obermiller, I., Shayandeh, S.: Appinsight: mobile app performance monitoring in the wild. In: 10th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2012, pp. 107–120 (2012)

  • Ravindranath, L., Padhye, J., Mahajan, R., Balakrishnan, H.: Timecard: controlling user-perceived delays in server-based mobile applications. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. SOSP ’13, pp. 85–100 (2013). https://doi.org/10.1145/2517349.2522717

  • Rosen, S., Nikravesh, A., Guo, Y., Mao, Z.M., Qian, F., Sen, S.: Revisiting network energy efficiency of mobile apps: performance in the wild. In: Proceedings of the 2015 ACM Internet Measurement Conference, IMC 2015, pp. 339–345 (2015). https://doi.org/10.1145/2815675.2815713

  • Telerik Fiddler: A Web Debugging Proxy Tool. https://www.telerik.com/fiddler/

  • Vásquez, M.L., Vendome, C., Luo, Q., Poshyvanyk, D.: How developers detect and fix performance bottlenecks in android apps. In: 2015 IEEE International Conference on Software Maintenance and Evolution, ICSME 2015, pp. 352–361 (2015). https://doi.org/10.1109/ICSM.2015.7332486

  • Wei, X., Gomez, L., Neamtiu, I., Faloutsos, M.: Profiledroid: multi-layer profiling of android applications. In: Proceedings of the 18th Annual International Conference on Mobile Computing and Networking. Mobicom ’12, pp. 137–148 (2012). https://doi.org/10.1145/2348543.2348563

  • Xiong, W., Chen, S., Zhang, Y., Xia, M., Qi, Z.: Reproducible interference-aware mobile testing. In: 2018 IEEE International Conference on Software Maintenance and Evolution, ICSME 2018, pp. 36–47 (2018). https://doi.org/10.1109/ICSME.2018.00013

  • Xu, Q., Mehrotra, S., Mao, Z., Li, J.: Proteus: network performance forecast for real-time, interactive mobile applications. In: Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services. MobiSys ’13, pp. 347–360 (2013). https://doi.org/10.1145/2462456.2464453

  • Zhang, L., Bild, D.R., Dick, R.P., Mao, Z.M., Dinda, P.: Panappticon: event-based tracing to measure mobile application and platform performance. In: Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis, CODES+ISSS 2013, pp. 1–10 (2013). https://doi.org/10.1109/CODES-ISSS.2013.6659020

  • Zhao, W., Ding, Z., Xia, M., Qi, Z.: Systematically testing and diagnosing responsiveness for android apps. In: 2019 IEEE International Conference on Software Maintenance and Evolution, pp. 449–453 (2019). https://doi.org/10.1109/ICSME.2019.00077

Download references

Acknowledgements

We would like to thank the anonymous reviewers of ICSME for their helpful comments. This work was sponsored by Shanghai Pujiang Program 19PJ1430900 and supported by the Shanghai Key Laboratory of Scalable Computing and Systems.

Author information

Author notes

  1. Zhanyao Lei and Wenhua Zhao contributed equally to this work.

Authors and Affiliations

  1. School of Software, Shanghai Jiao Tong University, Shanghai, China

    Zhanyao Lei, Wenhua Zhao, Zhenkai Ding & Zhengwei Qi

  2. AppetizerIO, Shanghai Bantang Inc., Shanghai, China

    Mingyuan Xia

Authors
  1. Zhanyao Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenhua Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenkai Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingyuan Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhengwei Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengwei Qi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, Z., Zhao, W., Ding, Z. et al. AppSPIN: reconfiguration-based responsiveness testing and diagnosing for Android Apps. Autom Softw Eng 29, 47 (2022). https://doi.org/10.1007/s10515-022-00347-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-022-00347-9

Keywords

Search

Navigation