skip to main content
10.1145/3629479.3629510acmotherconferencesArticle/Chapter ViewAbstractPublication PagessbqsConference Proceedingsconference-collections
research-article

Mock Objects in Software Testing: An Analysis of Usage in Open-Source Projects

Published: 06 December 2023 Publication History

Abstract

Dependencies in software are essential for efficiently structuring and modularizing the code, enabling software components to perform specific tasks, interact, and reuse functionalities. Furthermore, many applications also rely on external components, such as APIs and external services, which can difficult the creation of a testing environment. To overcome these challenges, developers can use mocks to simulate these dependencies, which becomes particularly useful when dealing with slow or hard-to-create dependencies, such as those requiring network access. Despite the use of mocks in software testing, there are few academic studies to understand the use of this technique. The main objective of this paper is to understand how mocks are used in automated tests in open-source projects, in addition to quantifying the use of support tools and assessing their impact on test creation. We discovered that Mockito, PHPUnit, Jest, and Mock tools are widely employed for Java, PHP, JavaScript, and Python programs, respectively. We also observed that the presence of mocks is consistent and follows the number of test files in each project. We noted that the external dependencies to the project were the most frequently simulated. However, we found no significant correlation between the number of mocks in the project and code coverage.

References

[1]
Maurício Aniche. 2015. Java code metrics calculator (CK). Available in https://github.com/mauricioaniche/ck/.
[2]
Andrea Arcuri, Gordon Fraser, and René Just. 2017. Private api access and functional mocking in automated unit test generation. In 2017 IEEE international conference on software testing, verification and validation (ICST). IEEE, 126–137.
[3]
Sebastian Bergmann. 2009. PHPLoc. Available in https://github.com/sebastianbergmann/phploc.
[4]
Hudson Borges, Andre Hora, and Marco Tulio Valente. 2016. Understanding the factors that impact the popularity of GitHub repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 334–344.
[5]
Tom Preston-Werner e Scott Chacon Chris Wanstrath, J. Hyett. 2019. Github website. https://github.com
[6]
Rômulo Martins da Silva, Cafer Cruz, Heleno de S. Campos, Leonardo GP Murta, and Vânia de Oliveira Neves. 2019. What is the adoption level of automated support for testing in open-source ecosystems?. In Proceedings of the IV Brazilian Symposium on Systematic and Automated Software Testing. 80–89.
[7]
Mattia Fazzini, Chase Choi, Juan Manuel Copia, Gabriel Lee, Yoshiki Kakehi, Alessandra Gorla, and Alessandro Orso. 2022. Use of test doubles in android testing: An in-depth investigation. In Proceedings of the 44th International Conference on Software Engineering. 2266–2278.
[8]
Martin Gruber and Gordon Fraser. 2022. A survey on how test flakiness affects developers and what support they need to address it. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 82–92.
[9]
Tim Mackinnon, Steve Freeman, and Philip Craig. 2000. Endo-testing: unit testing with mock objects. Extreme programming examined (2000), 287–301.
[10]
Avantika Mathur, Mingming Cao, and Andreas Dilger. 2007. ext4: the next generation of the ext3 file system. Usenix Association 32, 3 (2007), 25–30.
[11]
Gerard Meszaros. 2007. xUnit Test Patterns: Refactoring Test Code. Addison-Wesley, Upper Saddle River, NJ. https://www.safaribooksonline.com/library/view/xunit-test-patterns/9780131495050/
[12]
Shaikh Mostafa and Xiaoyin Wang. 2014. An empirical study on the usage of mocking frameworks in software testing. In 2014 14th international conference on quality software. IEEE, 127–132.
[13]
Gustavo Pereira and Andre Hora. 2020. Assessing mock classes: An empirical study. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 453–463.
[14]
Hesam Samimi, Rebecca Hicks, Ari Fogel, and Todd Millstein. 2013. Declarative mocking. In Proceedings of the 2013 International Symposium on Software Testing and Analysis. 246–256.
[15]
S Shaphiro and MBJB Wilk. 1965. An analysis of variance test for normality. Biometrika 52, 3 (1965), 591–611.
[16]
Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. 2017. To mock or not to mock? An empirical study on mocking practices. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 402ś412. https://doi. org/10.1109/MSR (2017).
[17]
Davide Spadini, Maurício Aniche, Magiel Bruntink, and Alberto Bacchelli. 2019. Mock objects for testing java systems: Why and how developers use them, and how they evolve. Empirical Software Engineering 24 (2019), 1461–1498.
[18]
Charles Spearman. 1904. The proof and measurement of association between two things. American journal of Psychology 15, 1 (1904), 72–101.
[19]
Terryin/lizard. 2009. Lizard - A simple code complexity analyser. Available in https://github.com/terryyin/lizard.
[20]
Fabian Trautsch and Jens Grabowski. 2017. Are there any unit tests? an empirical study on unit testing in open source python projects. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 207–218.
[21]
Andy Zaidman, Bart Van Rompaey, Serge Demeyer, and Arie Van Deursen. 2008. Mining software repositories to study co-evolution of production & test code. In 2008 1st international conference on software testing, verification, and validation. IEEE, 220–229.

Cited By

View all
  • (2024)Mimicking Production Behavior With Generated MocksIEEE Transactions on Software Engineering10.1109/TSE.2024.345844850:11(2921-2946)Online publication date: Nov-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SBQS '23: Proceedings of the XXII Brazilian Symposium on Software Quality
November 2023
391 pages
ISBN:9798400707865
DOI:10.1145/3629479
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Open Source
  2. Repository Mining
  3. Software Testing
  4. Testing Tools
  5. White-Box Coverage

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

SBQS '23
SBQS '23: XXII Brazilian Symposium on Software Quality
November 7 - 10, 2023
Bras\'{\i}lia, Brazil

Acceptance Rates

Overall Acceptance Rate 35 of 99 submissions, 35%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)44
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Mimicking Production Behavior With Generated MocksIEEE Transactions on Software Engineering10.1109/TSE.2024.345844850:11(2921-2946)Online publication date: Nov-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media