Alex Shye
ABSTRACT
Code coverage analysis, the process of finding code exercised by a particular set of test inputs, is an important component of software development and verification. Most traditional methods of implementing code coverage analysis tools are based on program instrumentation. These methods typically incur high overhead due to the insertion and execution of instrumentation code, and are not deployable in many software environments. Hardware-based sampling techniques attempt to lower overhead by leveraging existing Hardware Performance Monitoring (HPM) support for program counter (PC) sampling. While PC-sampling incurs lower levels of overhead, it does not provide complete coverage information. This paper extends the HPM approach in two ways. First, it utilizes the sampling of branch vectors which are supported on modern processors. Second, compiler analysis is performed on branch vectors to extend the amount of code coverage information derived from each sample. This paper shows that although HPM is generally used to guide performance improvement efforts, there is substantial promise in leveraging the HPM information for code debugging and verification. The combination of sampled branch vectors and compiler analysis can be used to attain upwards of 80% of the actual code coverage.
- A. Aho, R. Sethi, and J. Ullman. Compilers: Principles, Techniques and Tools. A-W Press, 1986. Google Scholar
Digital Library
- F. Allen. Control ow analysis. In Proceedings of Compiler Optimization, pages 1--19, 1970. Google ScholarDigital Library
- J. Anderson and et al. Continuous profiling: Where have all the cycles gone? In Proc. of the 16th ACM Symposium of Operating Systems Principles, pages 1--14, October 1997. Google ScholarDigital Library
- Bullseye Testing Technology. http://www.bullseye.com/.Google Scholar
- S. Eranian. The perfmon2 interface specification. Technical Report HPL-2004-200R1, Hewlett-Packard Laboratory, February 2005.Google Scholar
- C. L. et al. Pin: Building customized program analysis tools with dynamic instrumentation. In ACM SIGPLAN Conference on Programming Language Design and Implementation, Chicago, IL, June 2005. Google ScholarDigital Library
- Hewlett-Packard Development Company. perfmon project http://www.hpl.hp.com/research/linux/perfmon/.Google Scholar
- IBM. PowerPC 740/PowerPC 750 RISC Microprocessor User's Manual, 1999.Google Scholar
- IBM PureCoverage. http://www.pts.com/wp2077.cfm.Google Scholar
- Intel Corporation. Intel Itanium 2 processor reference manual: For software development and optimization. May 2004.Google Scholar
- M. Kearns and et al. On the learnability of discrete distributions. In STOC '94: Proceedings of the twenty-sixth annual ACM symposium on Theory of computing, pages 273--282. ACM Press, 1994. Google ScholarDigital Library
- Y. W. Kim. Efficient use of code coverage in large-scale software development. In CASCON '03: Proceedings of the 2003 conference of the Centre for Advanced Studies on Collaborative research, pages 145--155. IBM Press, 2003. Google ScholarDigital Library
- OpenIMPACT Compiler. http://www.gelato.uiuc.edu/.Google Scholar
- OProfile System Profiler for Linux. http://oprofile.sourceforge.net.Google Scholar
- A. Orso, N. Shi, and M. J. Harrold. Scaling regression testing to large software systems. In Proceedings of the 12th Symposium on Foundations of Software Engineering, pages 241--251. ACM Press, 2004. Google ScholarDigital Library
- B. Sprunt. Pentium 4 performance-monitoring features. In IEEE Micro 22(4), pages 72--82, 2002. Google ScholarDigital Library
- M. M. Tikir and J. K. Hollingsworth. Efficient instrumentation for code coverage testing. In Proceedings of the Symposium on Software Testing and Analysis, July 2002. Google ScholarDigital Library
- H. Zhu, P. A. V. Hall, and J. H. R. May. Software unit test coverage and adequacy. ACM Computing Survey, 29(4):366--427, 1997. Google ScholarDigital Library
Index Terms
- Code coverage testing using hardware performance monitoring support
Recommendations
Efficient instrumentation for code coverage testing
Evaluation of Code Coverage is the problem of identifying the parts of a program that did not execute in one or more runs of a program. The traditional approach for code coverage tools is to use static code instrumentation. In this paper we present a ...
Efficient instrumentation for code coverage testing
ISSTA '02: Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysisEvaluation of Code Coverage is the problem of identifying the parts of a program that did not execute in one or more runs of a program. The traditional approach for code coverage tools is to use static code instrumentation. In this paper we present a ...
Code Coverage Aware Test Generation Using Constraint Solver
Software Engineering and Formal Methods. SEFM 2020 Collocated WorkshopsAbstractCode coverage has been used in the software testing context mostly as a metric to assess a generated test suite’s quality. Recently, code coverage analysis is used as a white-box testing technique for test optimization. Most of the research ...
Comments