skip to main content
10.1145/1295074.1295085acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
Article

Generating combinatorial test suite for interaction relationship

Published: 03 September 2007 Publication History

Abstract

Combinatorial testing could detect the faults triggered by the interactions among factors in software. But in many cases, the pair-wise, N-way and even the variable strength combinatorial testing may lead test suite redundancy and fault detect ability decreasing, because these methods do not make sufficient consideration on the actual factors interaction. In this paper, a new interaction relationship based combinatorial testing model was proposed to cover the actual factor interactions in software by extending the conventional combinatorial testing model and IO relationship testing model. The new method may be more effectively than existed combinatorial testing methods without decrease of the fault detect ability. Furthermore, two test suite generation algorithms for interaction relationship based combinatorial testing were also presented. Finally, we compared our algorithms with some similar test generation algorithms in IO relationship testing model, and the experience result showed the advantage of our algorithms.

References

[1]
Cohen D M, Dalal S R, Fredman M L, et al. The AETG system: An approach to testing based on combinatorial design. IEEE Trans. on Software Engineering, 1997, 23, 7: 437--444.
[2]
Tai K C, Lei Y. A test generation strategy for pairwise testing. IEEE Trans. on Software Engineering, 2002, 28, 1: 109--111.
[3]
Tung Y W, Aldiwan W S. Automating Test Case Generation for the New Generation Mission Software System. In Proceedings of IEEE Arospace Conf., 2000, pp. 431--437.
[4]
Colbourn C J, Cohen M B, and Turban R C. A deterministic density algorithm for pairwise interaction coverage. In: Proceedings of IASTED International Conference on Software Engineering (SE2004), Innsbruck, Austria, 2004, 345--352.
[5]
Noritaka Kobayashi, Tatssuhio Tsuchiya, Tohru Kikuno. A new method for constructing pair-wise covering designs for software testing. Information Processing Letters, 2002, 81, 2: 85--91.
[6]
Williams A W. Software component interaction testing: Coverage measurement and generation of configurations. Ph.D Thesis, Ottawa-Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa, Canada, 2002.
[7]
Cohen M B, Colbouns C J, Ling A C H. Augmenting simulated annealing to build interaction test suites. In: Proceedings of 14th International Symposium on Software Reliability Engineering (ISSRE 2003), Denver Colorado, November 2003: 394--405.
[8]
Toshiaki Shiba, Tatsuhiro Tsuchiya, Tohru Kikuno. Using artificial life techniques to generate test cases for combinatorial testing. In: Proceedings of 28th International Computer Software and Applications Conference (COMPSAC2004), HongKong, China, 2004, 72--78.
[9]
Schroeder P J, Bolaki P, Gopu V. Comparing the fault detection effectiveness of n-way and random test suite. In: Proceedings of 2004 International Symposium on Empirical Software Engineering (ISESE2004), Redondo Beach, California, 2004, 49--59.
[10]
Yan Jun, Zhang Jian. Backtracking algorithms and search heuristics to generate test suites for combinatorial testing. In: Proceedings of 30th Annual International Conference on Computer Software and Applications (COMPSAC06), Volume 1, Sept. 2006:385 -- 394.
[11]
Yalmaz C, Cohen M B, Porter A A. Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans. on Software Engineering, 2006, 32, 1: 20--34.
[12]
Schroeder P J and Korel B. Black-box Test Reduction Using Input-output Analysis. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA'00), Portland, Oregon, Aug, 2000:21--22.
[13]
Schoeder P J, Black-box test reduction using input-output analysis. Ph.D. Thesis, Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA, 2001.
[14]
Schroeder P J, Faherty P, Korel B. Generating expected results for automated black-box testing. In: Proceedings of 17th IEEE International Conference on Automated Software Engineering (ASE'02).
[15]
Cheng C, Dumitrescu A, Schroeder P J. Generating small combinatorial test suites to cover input-output relationships. In: Proceedings of Third International Conference on Quality Software, Nov 2003: 76--82.
[16]
Bach J, Schroeder P J. Pairwise testing: a best practice that isn't. In: Proceedings of 22nd Pacific Northwest Software Quality Conference, 2004, 180--196.
[17]
Nie Changhai, Xu Baowen, Wang Ziyuan, Shi Liang. Generating optimal test set for neighbor factors combinatorial testing. In: Proceedings of Sixth International Conference on Quality Software, Oct 2006: 259--265.
[18]
Changhai Nie, Baowen Xu, Liang Shi, Guowei Dong. Automatic test generation for n-way combinatorial testing. In: Proceedings of Second International Workshop on Software Quality (SOQUA2005), Fair and Convention Center, Erfurt, Germany, 2005. Lecture Notes in Computer Science 3712, 2005: 203--211.
[19]
Nie Changhai, Xu Baowen, Shi Liang, Wang Ziyuan. A new heuristic for test suite generation for pairwise testing. In: Proceedings of 18th International Conference on Software Engineering and Knowledge Engineering (SEKE2006).
[20]
Kuhn D R, Reilly M J. An investigation of the applicability of design of experiments to software testing. In: Proceedings of 27th NASA/IEEE Software Engineering Workshop, NASA Goddard Space Flight Center, 2002, 91--95.
[21]
Kuhn D R, Wallace D R. Software fault interaction and implication for software testing. IEEE Trans. on Software Engineering, 2004, 30, 6: 1--4.

Cited By

View all
  • (2023)Variable-strength combinatorial testing of exported activities based on misexposure predictionJournal of Systems and Software10.1016/j.jss.2023.111773204(111773)Online publication date: Oct-2023
  • (2023)SCIPOG: Seeding and constraint support in IPOG strategy for combinatorial t-way testing to generate optimum test casesJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.11.01035:1(185-201)Online publication date: Jan-2023
  • (2022)Proposed Method of Seeding and Constraint in One-Parameter-At-a- Time Approach for t-way Testing2022 International Conference on Digital Transformation and Intelligence (ICDI)10.1109/ICDI57181.2022.10007210(39-45)Online publication date: 1-Dec-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SOQUA '07: Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting
September 2007
120 pages
ISBN:9781595937247
DOI:10.1145/1295074
  • General Chair:
  • Mauro Pezzè
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. combinatorial testing
  2. interaction relationship
  3. software testing
  4. test generation

Qualifiers

  • Article

Conference

ESEC/FSE07
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Variable-strength combinatorial testing of exported activities based on misexposure predictionJournal of Systems and Software10.1016/j.jss.2023.111773204(111773)Online publication date: Oct-2023
  • (2023)SCIPOG: Seeding and constraint support in IPOG strategy for combinatorial t-way testing to generate optimum test casesJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2022.11.01035:1(185-201)Online publication date: Jan-2023
  • (2022)Proposed Method of Seeding and Constraint in One-Parameter-At-a- Time Approach for t-way Testing2022 International Conference on Digital Transformation and Intelligence (ICDI)10.1109/ICDI57181.2022.10007210(39-45)Online publication date: 1-Dec-2022
  • (2021)An Efficient Combinatorial Input Output-Based Using Adaptive Firefly Algorithm with Elitism Relations TestingAdvances in Science, Technology and Engineering Systems Journal10.25046/aj0604266:4(223-232)Online publication date: Jul-2021
  • (2021)AutoCCAGProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00030(201-212)Online publication date: 22-May-2021
  • (2021)Hyper-Heuristic Strategy for Input-Output-Based Interaction TestingRecent Trends in Mechatronics Towards Industry 4.010.1007/978-981-33-4597-3_88(967-977)Online publication date: 16-Jul-2021
  • (2018)Evaluating Bees Algorithm for Sequence-based T-way Testing Test Data GenerationIndian Journal of Science and Technology10.17485/ijst/2018/v11i4/12108611:4(1-20)Online publication date: 1-Jan-2018
  • (2017)Design and Implementation of Combinatorial Testing Tools2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C)10.1109/QRS-C.2017.61(320-325)Online publication date: Jul-2017
  • (2016)Optimizing combinatorial input-output based relations testing using Ant Colony algorithm2016 3rd International Conference on Electronic Design (ICED)10.1109/ICED.2016.7804713(586-590)Online publication date: Aug-2016
  • (2016)The Design of Distributed Hardware Platform for Complex Electronic System TestingAdvances in Intelligent Information Hiding and Multimedia Signal Processing10.1007/978-3-319-50212-0_33(269-276)Online publication date: 22-Nov-2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media