research-article

A test suite reduction approach based on pairwise interaction of requirements

Authors:

Daoxu ChenAuthors Info & Claims

SAC '11: Proceedings of the 2011 ACM Symposium on Applied Computing

Pages 1390 - 1397

https://doi.org/10.1145/1982185.1982488

Published: 21 March 2011 Publication History

Abstract

Test suite reduction is one of the effective techniques to reduce the cost of regression testing. In particular, it tries to identify and remove redundant test cases according to a specific test coverage criterion. However, the excessive reduction in test cases may also significantly weaken the fault detection ability of the original test suite. In this paper, we conjecture that covering interaction of test requirements can improve the fault detection ability and propose a new test suite reduction approach. As a preliminary study, we firstly propose a pairwise interaction based coverage criterion (PWIC). Then we propose a pairwise interaction of requirements based test suite reduction approach (PWIR). To assess the feasibility and usefulness of our proposed approach, we implement PWIR approach and conduct an empirical study on seven real C programs. After analyzing the results of the empirical studies, we conclude that our approach can improve the fault detection ability without severely increasing the reduced test suite size.

References

[1]

J. Black, E. Melachrinoudis and D. Kaeli. Bi-criteria Models for All-uses Test Suite Reduction. In Proceedings of the 26th International Conference on Software Engineering, pp. 106--115, 2004.

Digital Library

[2]

T. Y. Chen and M. F. Lau. A New Heuristic for Test Suite Reduction. Information and Software Technology, 4(5--6): pp. 347--354, 1998.

[3]

Z. Y. Chen, B. W. Xu, X. F. Zhang and C. H. Nie. A Novel Approach for Test Suite Reduction Based on Requirement Relation Contraction. In Proceedings of the 2008 ACM Symposium on Applied Computing, pp. 390--394, 2008.

Digital Library

[4]

H. Do, S. Elbaum and G. Rothermel. Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact. Empirical Software Engineering, 10(4): pp. 405--435, 2005.

Digital Library

[5]

M. Harder, J. Mellen and M. D. Ernst. Improving Test Suites via Operational Abstraction. In Proceedings of the 25th International Conference on Software Engineering, pp. 60--71, 2003.

Digital Library

[6]

M. J. Harrold, R. Gupta and M. L. Soffa. A Methodology for Controlling the Size of a Test Suite. ACM Transactions on Software Engineering and Methodology, 2(3): pp. 270--285, 1993.

Digital Library

[7]

M. J. Harrold and A. Orso. Retesting Software During Development and Maintenance. In Frontiers of Software Maintenance, pp. 99--108, 2008.

[8]

M. Heimdahl and D. George. Test-Suite Reduction for Model Based Tests: Effectds on Test Quality and Implications for Testing. In Proceedings of 19th International Conference on Automated Software Engineering, pp. 176--185, 2004.

Digital Library

[9]

J. R. Horgan and S. London. Data Flow Coverage and the C Language. In Proceedings of the Symposium on Testing, Analysis and Verification, pp. 87--97, 1991.

Digital Library

[10]

H. Hsu and A. Orso. MINTS: A General Framework and Tool for Supporting Test-suite Minimization. In Proceedings of the 31st International Conference on Software Engineering, pp. 419--429, 2009.

Digital Library

[11]

D. Jeffrey and N. Gupta. Improving Fault Detection Capability by Selectively Retaining Test Cases During Test Suite Reduction. IEEE Transactions on Software Engineering, 33(2), pp. 108--123, 2007.

Digital Library

[12]

J. A. Jones and M. J. Harrold. Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage. IEEE Transactions on Software Engineering, 29(3), pp. 195--209, 2003.

Digital Library

[13]

M. Martina and A. Bertolino. Using Spanning Sets for Coverage Testing. IEEE Transactions on Software Engineering, 29(11), pp. 974--984, 2003.

Digital Library

[14]

S. McMaster and A. Memon. Call-Stack Coverage for GUI Test Suite Reduction. IEEE Transactions on Software Engineering, 34(1), pp. 99--115, 2008.

Digital Library

[15]

A. S. Namin and D. H. Andrews. The Influence of Size and Coverage on Test Suite Effectiveness. In Proceedings of the 10th International Symposium on Software Testing and Analysis, pp. 57--67, 2009.

Digital Library

[16]

M. Nashat and E. Khalid. Simulated Annealing and Genetic Algorithms for Optimal Regression Testing. Journal of Software Maintenance: Research and Practice. 11(1), pp. 19--34, 1999.

Digital Library

[17]

G. Rothermel, M. J. Harrold, J. Ostrin and C. Hong. An Empirical Study of the Effects of Minimization on the Fault-detection Capabilities of Test Suites. In Proceedings of the 14th International Conference on Software Maintenance, pp. 34--43, 1998.

Digital Library

[18]

S. Sprenkle, S. Sampath, E. Gibson, L. Pollock and A. Souter. An Empirical Comparison of Test Suite Reduction Techniques for User-Session-Based Testing of Web Applications. In Proceedings of the 21st IEEE International Conference on Software Maintenance, pp. 587--596, 2005.

Digital Library

[19]

S. Tallam and N. Gupta. A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization. In Proceedings of the 6th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, pp. 35--42, 2005.

Digital Library

[20]

B. Vaysburg, L. H. Tahat and B. Korel. Dependence Analysis in Reduction of Requirement based Test Suites. In Proceedings of the 3rd International Symposium on Software Testing and Analysis, pp. 107--111, 2002.

Digital Library

[21]

W. E. Wong, J. Horgan, S. London and A. Mathur. Effect of Test Set Minimization on Fault Detection Effectiveness. Software Practice and Experience. 28(4), pp. 347--369, 1998.

Digital Library

[22]

Y. B. Yu, J. A. Jones and M. J. Harrold. An Empirical Study of the Effects of Test-suite Reduction on Fault Localization. In Proceedings of the 30th International Conference on Software Engineering, pp. 201--210, 2008.

Digital Library

[23]

X. Zhang, Q. Gu, X. Chen, J. X. Qi and D. X. Chen. A Study of Relative Redundancy in Test-Suite Reduction While Retaining or Even Improving Fault-Localization. In Proceedings of the 25th Symposium On Applied Computing, pp. 2229--2236, 2010.

Digital Library

Cited By

Upreti KGhosh SVats PPatil HBhardwaj RGautam V(2024)A Comprehensive Analysis of Regression Test Case Assessment Using Humpback Whale OptimizationData Science and Big Data Analytics10.1007/978-981-99-9179-2_52(693-706)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-981-99-9179-2_52
Verma AChoudhary ATiwari S(2022)A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test casesInternational Journal of Information Technology10.1007/s41870-022-01031-715:2(1055-1068)Online publication date: 25-Jul-2022
https://doi.org/10.1007/s41870-022-01031-7
Agrawal AChoudhary AKaur A(2020)An Effective Regression Test Case Selection Using Hybrid Whale Optimization AlgorithmInternational Journal of Distributed Systems and Technologies10.4018/IJDST.202001010511:1(53-67)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.4018/IJDST.2020010105
Show More Cited By

Index Terms

A test suite reduction approach based on pairwise interaction of requirements
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Empirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity

Test suite reduction is an important approach that decreases the cost of regression testing. A test suite reduction technique operates based on the relationship between the test cases in the regression test suite and the test requirements in the program ...
Probability Model-Based Test Suite Reduction

The paper presents a test suite reduction approach based on probability models in regression testing. First, we model the process of software testing by using the joint probability of both test cases and program faults, and propose two probability ...
Improving the effectiveness of test suite reduction for user-session-based testing of web applications

Context: Test suite reduction is the problem of creating and executing a set of test cases that are smaller in size but equivalent in effectiveness to an original test suite. However, reduced suites can still be large and executing all the tests in a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '11: Proceedings of the 2011 ACM Symposium on Applied Computing

March 2011

1868 pages

ISBN:9781450301138

DOI:10.1145/1982185

Conference Chairs:
William Chu
Tunghai University, TaiChung, Taiwan
,
W. Eric Wong
University of Texas at Dallas, Richardson, Texas
,
Program Chairs:
Mathew J. Palakal
Indiana University Purdue University, Indianapolis
,
Chih-Cheng Hung
Southern Polytechnic State University, Marietta

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

SAC'11

Sponsor:

SIGAPP

SAC'11: The 2011 ACM Symposium on Applied Computing

March 21 - 24, 2011

TaiChung, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
274
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Upreti KGhosh SVats PPatil HBhardwaj RGautam V(2024)A Comprehensive Analysis of Regression Test Case Assessment Using Humpback Whale OptimizationData Science and Big Data Analytics10.1007/978-981-99-9179-2_52(693-706)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-981-99-9179-2_52
Verma AChoudhary ATiwari S(2022)A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test casesInternational Journal of Information Technology10.1007/s41870-022-01031-715:2(1055-1068)Online publication date: 25-Jul-2022
https://doi.org/10.1007/s41870-022-01031-7
Agrawal AChoudhary AKaur A(2020)An Effective Regression Test Case Selection Using Hybrid Whale Optimization AlgorithmInternational Journal of Distributed Systems and Technologies10.4018/IJDST.202001010511:1(53-67)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.4018/IJDST.2020010105
Chen YGopinath RTadakamalla AErnst MHolmes RFraser GAmmann PJust RGrundy JLe Goues CLo D(2020)Revisiting the relationship between fault detection, test adequacy criteria, and test set sizeProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering10.1145/3324884.3416667(237-249)Online publication date: 21-Dec-2020
https://dl.acm.org/doi/10.1145/3324884.3416667
Pomeranz I(2018)Dynamically Determined Preferred Values and a Design-for-Testability Approach for Multiplexer Select Inputs under Functional Test SequencesACM Transactions on Design Automation of Electronic Systems10.1145/321977823:5(1-16)Online publication date: 20-Aug-2018
https://dl.acm.org/doi/10.1145/3219778
Rehman Khan SLee SJavaid NAbdul W(2018)A Systematic Review on Test Suite Reduction: Approaches, Experiment’s Quality Evaluation, and GuidelinesIEEE Access10.1109/ACCESS.2018.28096006(11816-11841)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2809600
Agarwal PAgrawal A(2014)Fault-localization techniques for software systemsACM SIGSOFT Software Engineering Notes10.1145/2659118.265912539:5(1-8)Online publication date: 17-Sep-2014
https://dl.acm.org/doi/10.1145/2659118.2659125
Campos JAbreu R(2013)Leveraging a Constraint Solver for Minimizing Test SuitesProceedings of the 2013 13th International Conference on Quality Software10.1109/QSIC.2013.17(253-259)Online publication date: 29-Jul-2013
https://dl.acm.org/doi/10.1109/QSIC.2013.17
Khurana PTripathi AKushwaha D(2013)Change impact analysis and its regression test effort estimation2013 3rd IEEE International Advance Computing Conference (IACC)10.1109/IAdCC.2013.6514435(1420-1424)Online publication date: Feb-2013
https://doi.org/10.1109/IAdCC.2013.6514435

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten