Skip to main content
SpringerLink
Log in

Investigating types and survivability of performance bugs in mobile apps

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

A recent research showed that mobile apps represent nowadays 75% of the whole usage of mobile devices. This means that the mobile user experience, while tied to many factors (e.g., hardware device, connection speed, etc.), strongly depends on the quality of the apps being used. With “quality” here we do not simply refer to the features offered by the app, but also to its non-functional characteristics, such as security, reliability, and performance. This latter is particularly important considering the limited hardware resources (e.g., memory) mobile apps can exploit. In this paper, we present the largest study at date investigating performance bugs in mobile apps. In particular, we (i) define a taxonomy of the types of performance bugs affecting Android and iOS apps; and (ii) study the survivability of performance bugs (i.e., the number of days between the bug introduction and its fixing). Our findings aim to help researchers and apps developers in building performance-bugs detection tools and focusing their verification and validation activities on the most frequent types of performance bugs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://github.com/pcqpcq/open-source-android-apps

  2. https://github.com/dkhamsing/open-source-ios-apps

  3. For apps not published on the app stores we manually assigned the category by reading their description.

  4. https://developer.github.com/v3/

  5. Note that there is also a category “performance bad practices” in which we group issues related to high-level practices, i.e., issues due to the fact that developers do not apply performance best practices proposed by the platform designers.

  6. https://developer.android.com/reference/android/app/Activity.html

  7. https://github.com/Automattic/simplenote-android/issues/79

  8. https://www.apple.com/it/swift/

  9. https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ProgrammingWithObjectiveC/Introduction/Introduction.html

References

  • Ali M, Joorabchi ME, Mesbah A (2017) Same app, different app stores: a comparative study. In: International conference on mobile software engineering and systems (MOBILESoft), pp 79–90

  • Boehm BW, Brown JR, Lipow M (1976) Quantitative evaluation of software quality. In: Proceedings of the 2nd international conference on Software engineering. IEEE Computer Society Press, pp 592–605

  • Brebner PC (2016) Automatic performance modelling from application performance management (APM) data: an experience report. In: International conference on performance engineering (ICPE), pp 55–61

  • Carroll A, Heiser G (2010) An analysis of power consumption in a smartphone. In: USENIX Annual technical conference

  • Christensen R (2011) Plane Answers to Complex Questions: The Theory of Linear models, fouth edn. Springer Texts in Statistics. Springer

  • Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Earlbaum Associates

  • Conover WJ (1998) Practical Nonparametric Statistics, 3rd edn. Wiley, New York

  • Cortellessa V, Marco AD, Inverardi P (2011) Model-Based Software performance analysis. Springer, Berlin

  • Cruz L, Abreu R (2017) Performance-based guidelines for energy efficient mobile applications. In: International conference on mobile software engineering and systems (MOBILESoft), pp 46–57

  • Cox DR (1972) Regression models and life tables. J R Stat Soc B(34):187–202

    MathSciNet  MATH  Google Scholar 

  • Cruz L, Abreu R (2019) Catalog of energy patterns for mobile applications. Empir Software Eng 24:2209–2235. https://doi.org/10.1007/s10664-019-09682-0

    Article  Google Scholar 

  • Cumming G (2011) Introduction to the new Statistics:Effect sizes, confidence intervals, and Meta-Analysis. Routledge, Abingdon

  • Di Franco A, Guo H, Rubio-González C (2017) A comprehensive study of real-world numerical bug characteristics. In: International conference on automated software engineering (ASE), pp 509–519

  • Fazzini M, Orso A (2017) Automated cross-platform inconsistency detection for mobile apps. In: International conference on automated software engineering (ASE), pp 308–318

  • Fling B (2009) Mobile design and development: Practical concepts and techniques for creating mobile sites and Web apps. O’Reilly Media Inc., Sebastopol

  • Franke D, Weise C (2011) Providing a software quality framework for testing of mobile applications. In: International conference on software testing, verification and validation (ICST), pp 431–434

  • Gao Z, Bird C, Barr ET (2017) To type or not to type: quantifying detectable bugs in javascript. In: International conference on software engineering (ICSE), pp 758–769

  • Gegick M, Rotella P, Xie T (2010) Identifying security bug reports via text mining: an industrial case study. In: Working conference on mining software repositories (MSR), pp 11–20

  • Grechanik M, Fu C, Xie Q (2012) Automatically Finding Performance Problems with Feedback-directed Learning Software Testing. In: International conference on software engineering (ICSE), pp 156–166

  • Gregg B (2013) Systems performance: enterprise and the cloud. Pearson Education, London

  • Grissom RJ, Kim JJ (2005) Effect sizes for research: A broad practical approach, 2nd edn. Lawrence Earlbaum Associates

  • Gui J, Mcilroy S, Nagappan M, Halfond WGJ (2015) Truth in advertising: The hidden cost of mobile ads for software developers. In: International conference on software engineering (ICSE), pp 100–110

  • Guo C, Zhang J, Yan J, Zhang Z, Zhang Y (2013) Characterizing and detecting resource leaks in android applications. In: International conference on automated software engineering (ASE), pp 389–398

  • Hao S, Li D, Halfond WGJ, Govindan R (2012) Estimating Android applications’ CPU energy usage via Bytecode profiling. In: International workshop on green and sustainable software (GREENS), pp 1–7

  • Hao S, Li D, Halfond WGJ, Govindan R (2013) Estimating mobile application energy consumption using program analysis. In: ICSE’13, pp 92–101

  • Harrell FE Jr (2017) With contributions from Charles Dupont, many others.: Hmisc: Harrell Miscellaneous. https://CRAN.R-project.org/package=Hmisc. R package version 4.0-3

  • Harter DE, Krishnan MS, Slaughter SA (2000) Effects of process maturity on quality, cycle time, and effort in software product development. Manag Sci 46(4):451–466

    Article  Google Scholar 

  • Hecht G, Moha N, Rouvoy R (2016) An empirical study of the performance impacts of android code smells. In: International conference on mobile software engineering and systems (MOBILESoft), pp 59–69

  • Hedges LV, Olkin I (1985) Statistical methods for Meta-Analysis. Academic Press, Cambridge

  • Herzig K, Zeller A (2013) The impact of tangled code changes. In: Working conference on mining software repositories (MSR), pp 121–130

  • Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70

    MathSciNet  MATH  Google Scholar 

  • Incerto E, Tribastone M, Trubiani C (2018) Combined vertical and horizontal autoscaling through model predictive control. In: International conference on parallel and distributed computing (euro-par), pp 147–159

  • Jain R (1990) The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. Wiley, New York

  • Jin G, Song L, Shi X, Scherpelz J, Lu S (2012) Understanding and detecting real-world performance bugs. ACM SIGPLAN Not 47(6):77–88

    Article  Google Scholar 

  • Joorabchi ME, Ali M, Mesbah A (2015) Detecting inconsistencies in multi-platform mobile apps. In: International symposium on software reliability engineering (ISSRE), pp 450–460

  • Joorabchi ME, Mesbah A, Kruchten P (2013) Real challenges in mobile app development. In: International symposium on empirical software engineering and measurement (ESEM), pp 15–24

  • Jovic M, Adamoli A, Hauswirth M (2011) Catch me if you can: performance bug detection in the wild. In: ACM SIGPLAN Notices, vol 46, pp 155–170

  • Kan SH (2002) Metrics and models in software quality engineering. Addison-wesley Longman Publishing Co. Inc

  • Khalid H, Nagappan M, Hassan AE (2016) Examining the relationship between findbugs warnings and app ratings. IEEE Softw 33(4):34–39

    Article  Google Scholar 

  • Killian C, Nagaraj K, Pervez S, Braud R, Anderson JW, Jhala R (2010) Finding latent performance bugs in systems implementations. In: International symposium on foundations of software engineering (FSE), pp 17–26

  • Kleinbaum DG, Klein M (2005) Survival Analysis: A Self-Learning Text

  • Knoche H, Eichelberger H (2018) Using the raspberry pi and docker for replicable performance experiments: Experience paper. In: International conference on performance engineering (ICPE), pp 305–316

  • Lampson BW (2004) Computer security in the real world. Computer 37(6):37–46

    Article  Google Scholar 

  • Lee S, Heo M, Lee C, Kim M, Jeong G (2017) Applying deep learning based automatic bug triager to industrial projects. In: Joint meeting on foundations of software engineering (ESEC/FSE), pp 926–931

  • Li D, Hao S, Halfond WGJ, Govindan R (2013) Calculating source line level energy information for android applications. In: International symposium on software testing and analysis (ISSTA), pp 78–89

  • Li D, Hao S, Gui J, Halfond W (2014a) An empirical study of the energy consumption of Android applications. In: International conference on software maintenance and evolution (ICSME), pp 121–130

  • Li D, Jin Y, Sahin C, Clause J, Halfond W (2014b) Integrated energy-directed test suite optimization. In: International symposium on software testing and analysis (ISSTA), pp 339–350

  • Li D, Tran AH, Halfond W (2014c) Making web applications more energy efficient for OLED smartphones. In: International conference on software engineering (ICSE), pp 573–538

  • Li D, Tran AH, Halfond WGJ (2015a) Nyx: a display energy optimizer for mobile web apps. In: Joint meeting on foundations of software engineering (ESEC/FSE), pp 958–961

  • Li D, Halfond WGJ (2015b) Optimizing energy of http requests in android applications. In: International workshop on software development lifecycle for mobile (demobile), pp 25–28

  • Li D, Lyu Y, Gui J, Halfond W (2016) Automated energy optimization of http requests for mobile applications. In: International conference on software engineering (ICSE), pp 249–260

  • Lin Y, Radoi C, Dig D (2014) Retrofitting concurrency for android applications through refactoring. In: International symposium on foundations of software engineering (FSE), pp 341–352

  • Linares-Vȧsquez M, Bavota G, Bernal-cȧrdenas C, Oliveto R, Penta MD, Poshyvanyk D (2014) Mining energy-greedy API usage patterns in android apps: an empirical study. In: Working conference on mining software repositories (MSR), pp 2–11

  • Linares-Vȧsquez M, Bavota G, Bernal-cȧrdenas CE, Oliveto R, Penta MD, Poshyvanyk D (2015a) Optimizing energy consumption of guis in android apps: a multi-objective approach. In: Joint meeting on foundations of software engineering (ESEC/FSE), pp 143–154

  • Linares-Vásquez M, Vendome C, Luo Q, Poshyvanyk D (2015b) How developers detect and fix performance bottlenecks in android apps. In: International conference on software maintenance and evolution (ICSME), pp 352–361

  • Linares-Vȧsquez M, Vendome C, Tufano M, Poshyvanyk D (2017) How developers micro-optimize android apps. J Syst Softw 130:1–23

    Article  Google Scholar 

  • Linares-Vásquez M, Bavota G, Bernal-Cárdenas C, Penta MD, Oliveto R, Poshyvanyk D (2018) Multi-objective optimization of energy consumption of guis in android apps. ACM Trans Softw Eng Methodol 27(3):14:1–14:47

    Article  Google Scholar 

  • Liu Y, Xu C, Cheung SC (2014) Characterizing and detecting performance bugs for smartphone applications. In: International conference on software engineering (ICSE), pp 1013–1024

  • Madan BB, Gogeva-Popstojanova K, Vaidyanathan K, Trivedi KS (2002) Modeling and quantification of security attributes of software systems. In: International conference on dependable systems and networks (DSN), pp 505–514

  • Martin W, Sarro F, Jia Y, Zhang Y, Harman M (2017) A survey of app store analysis for software engineering. IEEE Trans Softw Eng 43(9):817–847

    Article  Google Scholar 

  • Mazuera-Rozo A, Bautista-Mora J, Linares-Vȧsquez M, Rueda S, Bavota G (2019a) The android OS stack and its vulnerabilities: an empirical study. Empir Softw Eng 24(4):2056–2101

    Article  Google Scholar 

  • Mazuera-Rozo A, Trubiani C, Linares-Vásquez M, Bavota G (2019b) Replication package. https://github.com/amazuerar/perf-bugs-mobile/

  • Mcilroy S, Shang W, Ali N, Hassan AE (2017) User reviews of top mobile apps in apple and google app stores. Commun ACM 60(11):62–67

    Article  Google Scholar 

  • Mondal M, Roy CK, Schneider KA (2017) Bug propagation through code cloning: an empirical study. In: International conference on software maintenance and evolution (ICSME), pp 227–237

  • Nagappan M, Shihab E (2016) Future trends in software engineering research for mobile apps. In: International conference on software analysis, evolution, and reengineering (SANER), vol 5, pp 21–32

  • Near JP, Jackson D (2016) Finding security bugs in web applications using a catalog of access control patterns. In: International conference on software engineering (ICSE), pp 947–958

  • Nistor A, Jiang T, Tan L (2013) Discovering, reporting, and fixing performance bugs. In: International working conference on mining software repositories (MSR), pp 237–246

  • Nistor A, Chang PC, Radoi C, Lu S (2015) CARAMEL: detecting and fixing performance problems that have non-intrusive fixes. In: International conference on software engineering (ICSE), pp 902–912

  • Noei E, Syer MD, Zou Y, Hassan AE, Keivanloo I (2017) A study of the relation of mobile device attributes with the user-perceived quality of android apps. Empir Softw Eng 22(6):3088–3116

    Article  Google Scholar 

  • Oliveira W, Oliveira R, Castor F (2017) A study on the energy consumption of android app development approaches. In: International conference on mining software repositories (MSR), pp 42–52

  • Olivo O, Dillig I, Lin C (2015) Static detection of asymptotic performance bugs in collection traversals. In: ACM SIGPLAN Notices, vol 50, pp 369–378

  • Panichella S, Panichella A, Beller M, Zaidman A, Gall HC (2016) The impact of test case summaries on bug fixing performance: an empirical investigation. In: International conference on software engineering (ICSE), pp 547–558

  • Parsons T, Murphy J (2008) Detecting performance antipatterns in component based enterprise systems. J Object Technol 7(3):55–91

    Article  Google Scholar 

  • Pathak A, Hu Y, Zhang M (2011a) Bootstrapping energy debugging on smartphones: a first look at energy bugs in mobile devices. In: Workshop on hot topics in networks (hotnets), pp Article no 5

  • Pathak A, Hu Y, Zhang M, Bahl P, Wang YM (2011b) Fine-grained power modeling for smartphones using system call tracing. In: European conference on computer systems (eurosys), pp 153–168

  • Pathak A, Hu Y, Zhang M (2012a) Where is the energy spent inside my app? fine grained energy accounting on smartphones with eprof. In: European conference on computer systems (eurosys), pp 29–42

  • Pathak A, Jindal A, Hu Y, Midkiff SP (2012b) What is keeping my phone awake? characterizing and detecting no-sleep energy bugs in smartphone apps. In: International conference on mobile systems, applications, and services (mobisys), pp 267–280

  • Ramakrishnan R, Kaur A (2017) Technique for detecting early-warning signals of performance deterioration in large scale software systems. In: International conference on performance engineering (ICPE), pp 213–222

  • Rodríguez RJ, Trubiani C, Merseguer J (2012) Fault-tolerant techniques and security mechanisms for model-based performance prediction of critical systems. In: International Symposium on Architecting Critical Systems, ISARCS, pp 21–30

  • Sahin C, Wan M, Tornquist P, McKenna R, Pearson Z, Halfond WGJ, Clause J (2016) How does code obfuscation impact energy usage? J Softw Evol Process 28,:565–588. https://doi.org/10.1002/smr.1762

    Article  Google Scholar 

  • Schulz H, Okanovic D, van Hoorn A, Ferme V, Pautasso C (2019) Behavior-driven load testing using contextual knowledge - approach and experiences. In: International conference on performance engineering (ICPE), pp 265–272

  • Selakovic M, Pradel M (2016) Performance issues and optimizations in javascript: an empirical study. In: International conference on software engineering (ICSE), pp 61–72

  • Sliwerski J, Zimmermann T, Zeller A (2005) When do changes induce fixes?. In: Proceedings of the 2005 International Workshop on Mining Software Repositories

  • Syer MD, Jiang ZM, Nagappan M, Hassan AE, Nasser M, Flora P (2014) Continuous validation of load test suites. In: International conference on performance engineering (ICPE), pp 259–270

  • Syer MD, Nagappan M, Hassan AE, Adams B (2013) Revisiting prior empirical findings for mobile apps: an empirical case study on the 15 most popular open-source android apps. In: Conference of the center for advanced studies on collaborative research, pp 283–297

  • Trivedi KS, Bobbio A (2017) Reliability and availability engineering - modeling, analysis, and applications. Cambridge University Press, Cambridge

  • Trubiani C, Bran A, van Hoorn A, Avritzer A, Knoche H (2018) Exploiting load testing and profiling for performance antipattern detection. Inf Softw Technol 95:329–345

    Article  Google Scholar 

  • Wan M, Jin Y, Li D, Halfond WGJ (2015) Detecting display energy hotspots in Android apps. In: International conference on software testing, verification and validation (ICST)

  • Wan Z, Lo D, Xia X, Cai L (2017) Bug characteristics in blockchain systems: a large-scale empirical study. In: International conference on mining software repositories (MSR), pp 413–424

  • Wang J, Dou W, Gao Y, Gao C, Qin F, Yin K, Wei J (2017) A comprehensive study on real world concurrency bugs in node.js. In: International conference on automated software engineering (ASE), pp 520–531

  • Wasserman AI (2010) Software engineering issues for mobile application development. In: International workshop on future of software engineering research (FSE/SDP), pp 397–400

  • Wert A, Happe J, Happe L (2013) Supporting swift reaction: Automatically uncovering performance problems by systematic experiments. In: International conference on software engineering (ICSE), pp 552–561

  • Williams LG, Smith CU (2003) Making the business case for software performance engineering. In: International conference on computer measurement group (CMG), pp 349–358

  • Woodside CM, Franks G, Petriu DC (2007) The future of software performance engineering. In: International workshop on the future of software engineering (FOSE), pp 171–187

  • Woodside CM (2015) Wosp-c’15: Workshop on challenges in performance methods for software development. In: International conference on performance engineering (ICPE), pp 349–350

  • Yang Y, Xiang P, Mantor M, Zhou H (2012) Fixing performance bugs: an empirical study of open-source gpgpu programs. In: International conference on parallel processing (ICPP), pp 329–339

  • Zaman S, Adams B, Hassan AE (2011) Security versus performance bugs: a case study on firefox. In: International conference on mining software repositories (MSR), pp 93–102

  • Zaman S, Adams B, Hassan AE (2012) A qualitative study on performance bugs. In: International conference of mining software repositories MSR, pp 199–208

  • Zhou Y, Sharma A (2017) Automated identification of security issues from commit messages and bug reports. In: Joint meeting on foundations of software engineering (ESEC/FSE), pp 914–919

Download references

Acknowledgements

Mazuera-Rozo and Bavota gratefully acknowledge the financial support of the Swiss National Science Foundation for the CCQR project (SNF Project No. 175513).

Author information

Authors and Affiliations

  1. Università della Svizzera Italiana, Lugano, Switzerland

    Alejandro Mazuera-Rozo & Gabriele Bavota

  2. Universidad de los Andes, Cundinamarca, Colombia

    Alejandro Mazuera-Rozo & Mario Linares-Vásquez

  3. Gran Sasso Science Institute, L’Aquila, Italy

    Catia Trubiani

Authors
  1. Alejandro Mazuera-Rozo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catia Trubiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Linares-Vásquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabriele Bavota
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandro Mazuera-Rozo.

Additional information

Communicated by: Meiyappan Nagappan

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mazuera-Rozo, A., Trubiani, C., Linares-Vásquez, M. et al. Investigating types and survivability of performance bugs in mobile apps. Empir Software Eng 25, 1644–1686 (2020). https://doi.org/10.1007/s10664-019-09795-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-019-09795-6

Keywords

Search

Navigation