Tássio Virgínio
Railana Santana
Luana Almeida Martins
Larissa Rocha Soares
ABSTRACT
Software testing is a key practice in the software quality assurance process. Usually, the quality of a test is not analyzed before its execution, i.e., there are no tests to check the tests. When the quality of tests is not guaranteed, it may impair the quality of the software. Test Smells are an alternative to indicate problems in the test code that can affect test maintainability, more specifically readability and comprehension. This study investigates correlations between test coverage and test smells types. We also introduce the JNose Test, a tool to automate test smells detection. We analyzed 11 open source projects and detected 21 types of smells and 10 different test coverage metrics to each test class. We identified 63 out of 210 calculated correlations. Our results show that there is a relationship between test smells and test coverage, in which test smells may influence code coverage. Our findings might support software testers and help them understand the behavior and consequences of poorly written and designed tests.
- Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering 20, 4 (2015).Google Scholar
Digital Library
- Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and David W. Binkley. 2012. An empirical analysis of the distribution of unit test smells and their impact on software maintenance. In 28th IEEE International Conference on Software Maintenance (ICSM).Google Scholar
- David Bowes, Tracy Hall, Jean Petrić, Thomas Shippey, and Burak Turhan. 2017. How Good Are My Tests?. In 2017 IEEE/ACM 8th Workshop on Emerging Trends in Software Metrics (WETSoM).Google ScholarCross Ref
- Adalberto Nobiato Crespo, Odair J Silva, Carlos Alberto Borges, Clênio Figueiredo Salviano, M Argollo, and Mario Jino. 2004. Uma metodologia para teste de Software no Contexto da Melhoria de Processo. In Simpósio Brasileiro de Qualidade de Software (SBQS).Google ScholarCross Ref
- Arie Deursen, Leon M.F. Moonen, A. Bergh, and Gerard Kok. 2001. Refactoring Test Code. Technical Report. Amsterdam, The Netherlands, The Netherlands.Google Scholar
- Gerald D. Everett and Raymond McLeod Jr. 2007. Automated Testing Tools (1 ed.). Wiley-IEEE Computer Society Press.Google Scholar
- Vahid Garousi and Banış Küçük. 2018. Smells in software test code: A survey of knowledge in industry and academia. Journal of Systems and Software 138 (2018).Google Scholar
- Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Code Coverage for Suite Evaluation by Developers. In Proceedings of the 36th International Conference on Software Engineering (ICSE).Google ScholarDigital Library
- Suhaimi Ibrahim and Shahid Muhammad. 2011. An Evaluation of Test Coverage Tools in Software Testing. In International Conference On Telecommunication Technology And Application (ICTTA).Google Scholar
- Gerard Meszaros. 2007. xUnit test patterns: refactoring test code (1st ed.). Addison-Wesley.Google Scholar
- Fabio Palomba, Dario Di Nucci, Annibale Panichella, Rocco Oliveto, and Andrea De Lucia. 2016. On the Diffusion of Test Smells in Automatically Generated Test Code: An Empirical Study. In IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST).Google ScholarDigital Library
- Fabio Palomba, Andy Zaidman, and Andrea De Lucia. 2018. Automatic Test Smell Detection Using Information Retrieval Techniques. In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).Google Scholar
- Anthony Peruma. 2018. What the Smell? An Empirical Investigation on the Distribution and Severity of Test Smells in Open Source Android Applications.Google Scholar
- Roger Pressman. 2016. Software Engineering: A Practitioner's Approach (8th ed.). McGraw-Hill, Inc., New York, NY.Google Scholar
- Bart Van Rompaey, Bart Du Bois, Serge Demeyer, and Matthias Rieger. 2007. On The Detection of Test Smells: A Metrics-Based Approach for General Fixture and Eager Test. IEEE Transactions on Software Engineering 33, 12 (Dec 2007).Google Scholar
- Neil J Salkind and Terese Rainwater. 2005. Exploring research (6 ed.). Prentice Hall.Google Scholar
- Davide Spadini, Fabio Palomba, Andy Zaidman, Magiel Bruntink, and Alberto Bacchelli. 2018. On the Relation of Test Smells to Software Code Quality. In IEEE International Conference on Software Maintenance and Evolution (ICSME).Google ScholarCross Ref
- Bharti Suri and Shweta Singhal. 2015. Investigating the OO characteristics of software using CKJM metrics. In 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions).Google ScholarCross Ref
- Michele Tufano, Fabio Palomba, Gabriele Bavota, Massimiliano Di Penta, Rocco Oliveto, Andrea De Lucia, and Denys Poshyvanyk. 2016. An Empirical Investigation into the Nature of Test Smells. In 31st IEEE/ACM Intern. Conf. on Automated Software Engineering (ASE). ACM, New York, NY, USA.Google ScholarDigital Library
- Vahid Garousi Yusifoğlu, Yasaman Amannejad, and Aysu Betin Can. 2015. Software test-code engineering: A systematic mapping. Information and Software Technology 58 (2015).Google Scholar
Index Terms
- On the influence of Test Smells on Test Coverage
Recommendations
Investigating Test Smells in JavaScript Test Code
SAST '21: Proceedings of the 6th Brazilian Symposium on Systematic and Automated Software TestingWriting automated test cases is a challenging and demanding activity. The test case itself is software that requires proper design to ensure it can be implemented and maintained as long as the production code evolves. Like code smells, test smells may ...
Program Segmentation for Controlling Test Coverage
ISSRE '97: Proceedings of the Eighth International Symposium on Software Reliability EngineeringIn this paper we present a new control-flow based approach to dynamic testing of sequential software. A practicable number of test cases is generated by using the boundary-interior path testing strategy and by dividing the test units into test segments (...
Multidimensional test coverage analysis: PARADIGM-COV tool
Currently, software tends to assume increasingly critical roles in our society so assuring its quality becomes ever more crucial. There are several tools and processes of software testing to help increase quality in virtually any type of software. One ...
Comments