Humberto Damasceno
ABSTRACT
Test smells represent a set of poorly designed tests, which can harm a test code’s maintenance and quality criteria. Although fundamental steps to understand test smells have been investigated, there is still an evident lack of studies evaluating the impact of test smell refactoring from internal quality attributes, such as size, cohesion, coupling, and complexity. In addition, the literature still lacks research that addresses the difficulties developers encounter during test smell refactoring. The paper investigates the impact of test smell refactoring from developers’ perspectives and internal quality attributes. We investigated the perceptions and difficulties encountered by 20 developers while removing 5 types of test smells in 4 open-source projects over two months. Through this experiment, we analyzed: (i) the impact that test smell refactoring has on internal quality attributes; (ii) developers’ perception of test smells as actual problems within a software system; and (iii) main difficulties encountered by developers during test smell refactoring. Our findings can help developers design a prioritization scheme for test smell refactoring and make them aware of the real benefits of test smell refactoring.
- Mamdouh Alenezi and Khaled Almustafa. 2015. Empirical analysis of the complexity evolution in open-source software systems. International Journal of Hybrid Information Technology 8, 2 (2015), 257–266.Google Scholar
Cross Ref
- Wajdi Aljedaani, Anthony Peruma, Ahmed Aljohani, Mazen Alotaibi, Mohamed Wiem Mkaouer, Ali Ouni, Christian D. Newman, Abdullatif Ghallab, and Stephanie Ludi. 2021. Test Smell Detection Tools: A Systematic Mapping Study. In Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE 2021). Association for Computing Machinery, New York, NY, USA, 170–180. https://doi.org/10.1145/3463274.3463335Google ScholarDigital Library
- Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and David Binkley. 2012. An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In 2012 28th IEEE International Conference on Software Maintenance (ICSM). 56–65. https://doi.org/10.1109/ICSM.2012.6405253Google ScholarDigital Library
- Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? an empirical study. Empirical Software Engineering 20, 4 (2015), 1052–1094. https://doi.org/10.1007/s10664-014-9313-0Google ScholarDigital Library
- Moritz Beller, Georgios Gousios, Annibale Panichella, Sebastian Proksch, Sven Amann, and Andy Zaidman. 2019. Developer Testing in the IDE: Patterns, Beliefs, and Behavior. IEEE Transactions on Software Engineering 45, 3 (2019), 261–284. https://doi.org/10.1109/TSE.2017.2776152Google ScholarDigital Library
- Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman. 2015. When, How, and Why Developers (Do Not) Test in Their IDEs. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE 2015). Association for Computing Machinery, New York, NY, USA, 179–190. https://doi.org/10.1145/2786805.2786843Google ScholarDigital Library
- S. Berner, R. Weber, and R.K. Keller. 2005. Observations and lessons learned from automated testing. In Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005.571–579. https://doi.org/10.1109/ICSE.2005.1553603Google ScholarCross Ref
- Antonia Bertolino. 2007. Software Testing Research: Achievements, Challenges, Dreams. In Future of Software Engineering (FOSE ’07). 85–103. https://doi.org/10.1109/FOSE.2007.25Google ScholarDigital Library
- James M. Bieman and Byung-Kyoo Kang. 1995. Cohesion and Reuse in an Object-Oriented System. SIGSOFT Softw. Eng. Notes 20, SI (aug 1995), 259–262. https://doi.org/10.1145/223427.211856Google ScholarDigital Library
- Denivan Campos, Larissa Rocha, and Ivan Machado. 2021. Developers perception on the severity of test smells: an empirical study. arXiv preprint arXiv:2107.13902(2021). https://doi.org/10.48550/arXiv.2107.13902Google ScholarCross Ref
- George Candea, Stefan Bucur, and Cristian Zamfir. 2010. Automated Software Testing as a Service. In Proceedings of the 1st ACM Symposium on Cloud Computing (Indianapolis, Indiana, USA) (SoCC ’10). Association for Computing Machinery, New York, NY, USA, 155–160. https://doi.org/10.1145/1807128.1807153Google ScholarDigital Library
- Alexander Chávez, Isabella Ferreira, Eduardo Fernandes, Diego Cedrim, and Alessandro Garcia. 2017. How Does Refactoring Affect Internal Quality Attributes? A Multi-Project Study. In Proceedings of the 31st Brazilian Symposium on Software Engineering (Fortaleza, CE, Brazil) (SBES’17). Association for Computing Machinery, New York, NY, USA, 74–83. https://doi.org/10.1145/3131151.3131171Google ScholarDigital Library
- Shyam R. Chidamber and Chris F. Kemerer. 1991. Towards a Metrics Suite for Object Oriented Design. SIGPLAN Not. 26, 11 (nov 1991), 197–211. https://doi.org/10.1145/118014.117970Google ScholarDigital Library
- D.P. Darcy, C.F. Kemerer, S.A. Slaughter, and J.E. Tomayko. 2005. The structural complexity of software an experimental test. IEEE Transactions on Software Engineering 31, 11 (2005), 982–995. https://doi.org/10.1109/TSE.2005.130Google ScholarDigital Library
- Robert Dyer, Hridesh Rajan, and Yuanfang Cai. 2012. An Exploratory Study of the Design Impact of Language Features for Aspect-Oriented Interfaces. In Proceedings of the 11th Annual International Conference on Aspect-Oriented Software Development (Potsdam, Germany) (AOSD ’12). Association for Computing Machinery, New York, NY, USA, 143–154. https://doi.org/10.1145/2162049.2162067Google ScholarDigital Library
- Eduardo Fernandes, Alexander Chávez, Alessandro Garcia, Isabella Ferreira, Diego Cedrim, Leonardo Sousa, and Willian Oizumi. 2020. Refactoring effect on internal quality attributes: What haven’t they told you yet?Information and Software Technology 126 (2020), 106347. https://doi.org/10.1016/j.infsof.2020.106347Google ScholarCross Ref
- Martin Fowler. 1999. Refactoring: improving the design of existing code. ilustrated edition.Google Scholar
- César França, Fabio Q. B. da Silva, and Helen Sharp. 2020. Motivation and Satisfaction of Software Engineers. IEEE Transactions on Software Engineering 46, 2 (2020), 118–140. https://doi.org/10.1109/TSE.2018.2842201Google ScholarCross Ref
- ISO. 2011. IEC 25010: 2011 systems and software engineering–systems and software quality requirements and evaluation (square)–system and software quality models. International Organization for Standardization 34 (2011), 2910.Google Scholar
- Amandeep Kaur and Gaurav Dhiman. 2019. A review on search-based tools and techniques to identify bad code smells in object-oriented systems. In Harmony search and nature inspired optimization algorithms. Springer, 909–921. https://doi.org/10.1007/978-981-13-0761-4_86Google ScholarCross Ref
- Dong Jae Kim. 2020. An Empirical Study on the Evolution of Test Smell. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings (Seoul, South Korea) (ICSE ’20). Association for Computing Machinery, New York, NY, USA, 149–151. https://doi.org/10.1145/3377812.3382176Google ScholarDigital Library
- Dong Jae Kim, Tse-Hsun (Peter) Chen, and Jinqiu Yang. 2021. The Secret Life of Test Smells - an Empirical Study on Test Smell Evolution and Maintenance. Empirical Softw. Engg. 26, 5 (sep 2021), 47 pages. https://doi.org/10.1007/s10664-021-09969-1Google ScholarDigital Library
- Mark Lorenz and Jeff Kidd. 1994. Object-oriented software metrics: a practical guide. Prentice-Hall, Inc.Google ScholarDigital Library
- Ruchika Malhotra and Anuradha Chug. 2016. An empirical study to assess the effects of refactoring on software maintainability. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI). 110–117. https://doi.org/10.1109/ICACCI.2016.7732033Google ScholarCross Ref
- Júlio Martins, Carla Bezerra, Anderson Uchôa, and Alessandro Garcia. 2021. How Do Code Smell Co-Occurrences Removal Impact Internal Quality Attributes? A Developers’ Perspective. In Brazilian Symposium on Software Engineering(Joinville, Brazil) (SBES ’21). Association for Computing Machinery, New York, NY, USA, 54–63. https://doi.org/10.1145/3474624.3474642Google ScholarDigital Library
- T.J. McCabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering SE-2, 4 (1976), 308–320. https://doi.org/10.1109/TSE.1976.233837Google ScholarDigital Library
- Sandro Morasca. 2009. A probability-based approach for measuring external attributes of software artifacts. In 2009 3rd International Symposium on Empirical Software Engineering and Measurement. 44–55. https://doi.org/10.1109/ESEM.2009.5316048Google ScholarDigital Library
- Glenford J Myers, Corey Sandler, and Tom Badgett. 2011. The art of software testing. John Wiley & Sons.Google ScholarDigital Library
- Alessandro Orso and Gregg Rothermel. 2014. Software Testing: A Research Travelogue (2000–2014). In Future of Software Engineering Proceedings (Hyderabad, India) (FOSE 2014). Association for Computing Machinery, New York, NY, USA, 117–132. https://doi.org/10.1145/2593882.2593885Google ScholarDigital Library
- Matheus Paixão, Anderson Uchôa, Ana Carla Bibiano, Daniel Oliveira, Alessandro Garcia, Jens Krinke, and Emilio Arvonio. 2020. Behind the Intents: An In-Depth Empirical Study on Software Refactoring in Modern Code Review. Association for Computing Machinery, New York, NY, USA, 125–136. https://doi.org/10.1145/3379597.3387475Google ScholarDigital Library
- Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, and Andrea De Lucia. 2014. Do They Really Smell Bad? A Study on Developers’ Perception of Bad Code Smells. In 2014 IEEE International Conference on Software Maintenance and Evolution. 101–110. https://doi.org/10.1109/ICSME.2014.32Google ScholarDigital Library
- Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 311–322. https://doi.org/10.1109/ICSME.2018.00040Google ScholarCross Ref
- Fabiano Pecorelli, Fabio Palomba, and Andrea De Lucia. 2021. The relation of test-related factors to software quality: A case study on apache systems. Empirical Software Engineering 26, 2 (2021), 1–42. https://doi.org/10.1007/s10664-020-09891-yGoogle ScholarDigital Library
- Anthony Peruma, Khalid Saeed Almalki, Christian D Newman, Mohamed Wiem Mkaouer, Ali Ouni, and Fabio Palomba. 2019. On the distribution of test smells in open source android applications: An exploratory study. (2019). https://dl.acm.org/doi/10.5555/3370272.3370293Google Scholar
- Elvys Soares, Márcio Ribeiro, Guilherme Amaral, Rohit Gheyi, Leo Fernandes, Alessandro Garcia, Baldoino Fonseca, and André Santos. 2020. Refactoring Test Smells: A Perspective from Open-Source Developers. In Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing (Natal, Brazil) (SAST 20). Association for Computing Machinery, New York, NY, USA, 50–59. https://doi.org/10.1145/3425174.3425212Google ScholarDigital Library
- Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME). 1–12. https://doi.org/10.1109/ICSME.2018.00010Google ScholarCross Ref
- Davide Spadini, Martin Schvarcbacher, Ana-Maria Oprescu, Magiel Bruntink, and Alberto Bacchelli. 2020. Investigating Severity Thresholds for Test Smells. In Proceedings of the 17th International Conference on Mining Software Repositories (Seoul, Republic of Korea) (MSR ’20). Association for Computing Machinery, New York, NY, USA, 311–321. https://doi.org/10.1145/3379597.3387453Google ScholarDigital Library
- Amjed Tahir, Steve Counsell, and Stephen G. MacDonell. 2016. An Empirical Study into the Relationship Between Class Features and Test Smells. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC). 137–144. https://doi.org/10.1109/APSEC.2016.029Google ScholarCross Ref
- Sandhya Tarwani and Anuradha Chug. 2016. Sequencing of refactoring techniques by Greedy algorithm for maximizing maintainability. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI). 1397–1403. https://doi.org/10.1109/ICACCI.2016.7732243Google ScholarCross Ref
- Anderson Uchôa, Caio Barbosa, Daniel Coutinho, Willian Oizumi, Wesley K. G. Assunção, Silvia Regina Vergilio, Juliana Alves Pereira, Anderson Oliveira, and Alessandro Garcia. 2021. Predicting Design Impactful Changes in Modern Code Review: A Large-Scale Empirical Study. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 471–482. https://doi.org/10.1109/MSR52588.2021.00059Google ScholarCross Ref
- Arie Van Deursen, Leon Moonen, Alex Van Den Bergh, and Gerard Kok. 2001. Refactoring test code. In Proceedings of the 2nd international conference on extreme programming and flexible processes in software engineering (XP2001). Citeseer, 92–95.Google Scholar
- Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On The Detection of Test Smells: A Metrics-Based Approach for General Fixture and Eager Test. IEEE Transactions on Software Engineering 33, 12 (2007), 800–817. https://doi.org/10.1109/TSE.2007.70745Google ScholarDigital Library
- Tássio Virgínio, Luana Martins, Larissa Rocha, Railana Santana, Adriana Cruz, Heitor Costa, and Ivan Machado. 2020. JNose: Java Test Smell Detector. In Proceedings of the 34th Brazilian Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for Computing Machinery, New York, NY, USA, 564–569. https://doi.org/10.1145/3422392.3422499Google ScholarDigital Library
- Guowu Xie, Jianbo Chen, and Iulian Neamtiu. 2009. Towards a better understanding of software evolution: An empirical study on open source software. In 2009 IEEE International Conference on Software Maintenance. 51–60. https://doi.org/10.1109/ICSM.2009.5306356Google ScholarCross Ref
- Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015), 123–147. https://doi.org/10.1016/j.infsof.2014.06.009Google ScholarCross Ref
Index Terms
- Analyzing Test Smells Refactoring from a Developers Perspective
Recommendations
Prioritising Refactoring Using Code Bad Smells
ICSTW '11: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation WorkshopsWe investigated the relationship between six of Fowler et al.'s Code Bad Smells (Duplicated Code, Data Clumps, Switch Statements, Speculative Generality, Message Chains, and Middle Man) and software faults. In this paper we discuss how our results can ...
The smell of fear: on the relation between test smells and flaky tests
AbstractRegression testing is the activity performed by developers to check whether new modifications have not introduced bugs. A crucial requirement to make regression testing effective is that test cases are deterministic. Unfortunately, this is not ...
Are test smells really harmful? An empirical study
Bad code smells have been defined as indicators of potential problems in source code. Techniques to identify and mitigate bad code smells have been proposed and studied. Recently bad test code smells (test smells for short) have been put forward as a ...
Comments