Akash Kumar Tripathi
ABSTRACT
This paper presents the results of an experimental study, in which four open source static program analysis tools namely, FindBugs, CodePro Analytix, UCDetector, and PMD have been applied on four small Java projects to appraise the bug detection effectiveness and efficiency on mutation bugs. In this experiment, we generated multiple applicable mutants of each projects using MuJava tool and subsequently applied the four SPA tools to measure the bug detecting effectiveness and efficiency. The obtained data was analyzed on two different bug classifications, namely mutant based and severity based. Our results showed that PMD demonstrated the maximum bug detecting effectiveness as well as efficiency among the four SPA tools, whereas CodePro Analytix identified bugs at all category of severity including most of the high severity bugs.
- Findbugs. http://findbugs.sourceforge.net/. Accessed: 2013-12-20.Google Scholar
- Mujava. http://cs.gmu.edu/~offutt/mujava/. Accessed: 2013-12-20.Google Scholar
- Bus. http://1000projects.org/bus-scheduling-and-booking-system-cse-java-project-withcode.html.Google Scholar
- Car. http://projectabstracts.com/1798/simple-car-sales-system-in-java.html.Google Scholar
- CodePro. http://wiki.eclipse.org/imgaes/7/75/CodeProDatasheet.pdf/.Google Scholar
- Hotel. http://sourceforge.net/p/hotelmgmtsys/code/HEAD/tree/.Google Scholar
- Y.-S. Ma, J. Offutt, and Y.-R. Kwon. Mujava: A mutation system for java. In Proceedings of the 28th International Conference on Software Engineering, ICSE '06, pages 827--830, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- N. Meng, Q. Wang, Q. Wu, and H. Mei. An approach to merge results of multiple static analysis tools (short paper). In Quality Software, 2008. QSIC'08. The Eighth International Conference on, pages 169--174. IEEE, 2008. Google ScholarDigital Library
- Monopoly. https://code.google.com/p/cosc603rajendranmonopoly/source/browse/trunk/+cosc603rajendranmonopoly/Monopoly/?r=9.Google Scholar
- Pmd. http://pmd.sourceforge.net/.Google Scholar
- N. Rutar, C. B. Almazan, and J. S. Foster. A comparison of bug finding tools for java. In Software Reliability Engineering, 2004. ISSRE 2004. 15th International Symposium on, pages 245--256. IEEE, 2004. Google ScholarDigital Library
- UCDetector. http://www.ucdetector.org/.Google Scholar
- S. Wagner, J. Jürjens, C. Koller, and P. Trischberger. Comparing bug finding tools with reviews and tests. In Proceedings of the 17th IFIP TC6/WG 6.1 International Conference on Testing of Communicating Systems, TestCom'05, pages 40--55, Berlin, Heidelberg, 2005. Springer-Verlag. Google ScholarDigital Library
- M. S. Ware and C. J. Fox. Securing java code: heuristics and an evaluation of static analysis tools. In Proceedings of the 2008 workshop on Static analysis, pages 12--21. ACM, 2008. Google ScholarDigital Library
Index Terms
- A controlled experiment to evaluate the effectiveness and the efficiency of four static program analysis tools for Java programs
Recommendations
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and AnalysisEmpirical studies in software testing research may not be comparable, reproducible, or characteristic of practice. One reason is that real bugs are too infrequently used in software testing research. Extracting and reproducing real bugs is challenging ...
A controlled experiment of different code representations for learning-based program repair
AbstractTraining a deep learning model on source code has gained significant traction recently. Since such models reason about vectors of numbers, source code needs to be converted to a code representation before vectorization. Numerous approaches have ...
A Controlled Experiment to Evaluate Effectiveness and Efficiency of Three Software Testing Methods
ICST '13: Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and ValidationIn this paper, we describe a controlled experiment carried out to compare three software testing methods: code reading, functional testing and structural testing. The experiment was carried out with eighteen subjects who applied three techniques to ...
Comments