skip to main content
10.1145/1287624.1287706acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
Article

Quantifying the effectiveness of testing via efficient residual path profiling

Published: 07 September 2007 Publication History

Abstract

Software testing is extensively used for uncovering bugs inlarge, complex software. Testing relies on well designed regression test suites that anticipate all reasonable software usage scenarios. Unfortunately, testers today have no way of knowing how much of real-world software usage was untested by their regression suite. Recent advances in low-overhead path profiling provide the opportunity to rectify this deficiency and perform residual path profiling on deployed software. Residual path profiling identifies all paths executed bydeployed software that were untested during software development. We extend prior research to perform low-overhead interprocedural path profiling. We demonstrate experimentally that low-overhead path profiling, both intraprocedural and interprocedural, provides valuable quantitative information on testing effectiveness. We also show that residual edge profiling is inadequate as a significant number of untested paths include no new untested edges.

References

[1]
T. Ball and J. R. Larus. Efficient path profiling. In International Symposium on Microarchitecture (MICRO), pages 46--57, 1996.
[2]
B. Beiser. Software testing techniques. Van Nostram Reinhold Inc., N. Y., 1990.
[3]
T. Chilimbi, A. V. Nori, and K. Vaswani. Quantifying the effectiveness of testing via efficient residual path profiling. Technical Report TR-2007-62, Microsoft Research, 2007.
[4]
J. R. Larus. Whole program paths. In ACM SIGPLAN Symposium on Programming Language Design and Implementation (PLDI), pages 259--269, 1999.
[5]
K. S. McKinley, J. Burrill, M. D. Bond, D. Burger, B. Cahoon, J. Gibson, J. E. B. Moss, A. Smith, Z.Wang, and C. Weems. The Scale compiler. http://ali-www.cs.umass.edu/Scale, 2005.
[6]
D. Melski and T. W. Reps. Interprocedural path profiling. In Proceedings of the 8th International Conference on Compiler Construction (CC), pages 47--62, 1999.
[7]
C. Pavlopoulou and M. Young. Residual test coverage monitoring. In Proceedings of the International conference on Software engineering (ICSE), pages 277--284, 1999.
[8]
S. Tallam, X. Zhang, and R. Gupta. Extending path profiling across loop backedges and procedure boundaries. In International Symposium on Code Generation and Optimization (CGO), pages 251--264, 2004.
[9]
K. Vaswani, A. V. Nori, and T. M. Chilimbi. Preferential path profiling: Compactly numbering interesting paths. In ACM SIGPLAN Conference on Principles of Programming Languages (POPL), pages 351--362, 2007.

Cited By

View all
  • (2015)P3: partitioned path profilingProceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering10.1145/2786805.2786868(485-495)Online publication date: 30-Aug-2015
  • (2013)SchnauzerProceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)10.1109/CGO.2013.6494998(1-11)Online publication date: 23-Feb-2013
  • (2013)Pertinent path profilingProceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)10.1109/CGO.2013.6494983(1-12)Online publication date: 23-Feb-2013
  • Show More Cited By

Index Terms

  1. Quantifying the effectiveness of testing via efficient residual path profiling

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering
    September 2007
    638 pages
    ISBN:9781595938114
    DOI:10.1145/1287624
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 September 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. inter-procedural
    2. path profiling
    3. residual
    4. testing

    Qualifiers

    • Article

    Conference

    ESEC/FSE07
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 112 of 543 submissions, 21%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2015)P3: partitioned path profilingProceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering10.1145/2786805.2786868(485-495)Online publication date: 30-Aug-2015
    • (2013)SchnauzerProceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)10.1109/CGO.2013.6494998(1-11)Online publication date: 23-Feb-2013
    • (2013)Pertinent path profilingProceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)10.1109/CGO.2013.6494983(1-12)Online publication date: 23-Feb-2013
    • (2009)HOLMESProceedings of the 31st International Conference on Software Engineering10.1109/ICSE.2009.5070506(34-44)Online publication date: 16-May-2009

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media