skip to main content
10.1145/3167132.3167335acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

Generating minimal test set satisfying MC/DC criterion via SAT based approach

Published: 09 April 2018 Publication History

Abstract

The Modified Condition/Decision Coverage (MC/DC) is a test criterion proposed by NASA to measure the test set adequacy, especially in safety-critical systems. Existing works on test case generation for this criterion usually make use of greedy or meta-heuristic search techniques to construct a small test set. This paper studies the problem of generating optimal test set satisfying this criterion. It enumerates the size of test set from the theoretic lower-bound m + 1 to upper-bound 2m where m is the number of conditions that can be MC/DC covered, and then employs a SAT solver to decide whether there exists a test set with a given size, until a test set is found. We apply this method to benchmarks with no more than 25 conditions including dozens of instances collected from related works and millions of randomly generated ones. The experimental results indicate that most (more than 99%) of these instances can be MC/DC covered by a test set with only m + 1 test cases. Therefore, our SAT-based approach is efficient for generating optimal MC/DC test sets, since for most of the instances, it only invokes the SAT solver just once in the process of searching for minimal test set.

References

[1]
Zeina Awedikian, Kamel Ayari, and Giuliano Antoniol. 2009. MC/DC automatic test input data generation. In Genetic and Evolutionary Computation Conference, GECCO 2009, Proceedings, Montreal, Québec, Canada, July 8-12, 2009.
[2]
A. Biere, M. Heule, H. van Maaren, and T. Walsh. 2009. Handbook of Satisfiability: Volume 185 Frontiers in Artificial Intelligence and Applications. IOS Press, Amsterdam, The Netherlands.
[3]
Pierre Flener, Alan M. Frisch, Brahim Hnich, Zeynep Kiziltan, Ian Miguel, Justin Pearson, and Toby Walsh. 2002. Breaking Row and Column Symmetries in Matrix Models. In Proceedings of the 8th International Conference on Principles and Practice of Constraint Programming (CP '02). Springer-Verlag, London, UK, 462--476.
[4]
Kamran Ghani and John A. Clark. 2009. Automatic Test Data Generation for Multiple Condition and MCDC Coverage. In The Fourth International Conference on Software Engineering Advances, ICSEA 2009, 20-25 September 2009, Porto, Portugal. 152--157.
[5]
Kelly J. Hayhurst, Dan S. Veerhusen, John J. Chilenski, and Leanna K. Rierson. 2001. A Practical Tutorial on Modified Condition/Decision Coverage. Technical Report. NASA Langley Technical Report Server, NASA/TM-2001-210876.
[6]
Sekou Kangoye, Alexis Todoskoff, Mihaela Barreau, and Philippe Germanicus. 2015. MC/DC Test Case Generation Approaches for Decisions. In Proceedings of the 24th Australasian Software Engineering Conference, ASWEC 2015, Volume II, Adelaide, SA, Australia, September 28 - October 1, 2015. 74--80.
[7]
CAST-10 Paper. 2002. What is a "Decision" in Application of Modified Condition/Decision Coverage (MC/DC) and Decision Coverage (DC)?
[8]
RTCA/DO-178B. 1992. Software Considerations in Airborne Systems and Equipment Certification. RTCA, Inc.
[9]
G. S. Tseitin. 1983. On the Complexity of Derivation in Propositional Calculus. Springer Berlin Heidelberg, Berlin, Heidelberg, 466--483.
[10]
Elaine J. Weyuker, Tarak Goradia, and Ashutosh Singh. 1994. Automatically Generating Test Data from a Boolean Specification. IEEE Trans. Software Eng. 20, 5 (1994), 353--363.
[11]
Tianyong Wu, Jun Yan, and Jian Zhang. 2014. Automatic Test Data Generation for Unit Testing to Achieve MC/DC Criterion. In Eighth International Conference on Software Security and Reliability, SERE 2014, San Francisco, California, USA, June 30 - July 2, 2014. 118--126.
[12]
Jun Yan and Jian Zhang. 2006. SAT based automated test case generation for MUMCUT coverage. The 17th IEEE International Symposium on Software Reliability Engineering (Student Program Papers) ISSRE (2006).
[13]
Yuen-Tak Yu and Man Fai Lau. 2006. A comparison of MC/DC, MUMCUT and several other coverage criteria for logical decisions. Journal of Systems and Software 79, 5 (2006), 577--590.

Cited By

View all
  • (2024)Is MCDC Really Better? Lessons from Combining Tests and ProofsTests and Proofs10.1007/978-3-031-72044-4_2(25-44)Online publication date: 10-Sep-2024
  • (2022)Towards an Automatic Test Generation Method for Systems of Systems Based on Fault Injection and Model-Based Systems EngineeringApplied Sciences10.3390/app12221186312:22(11863)Online publication date: 21-Nov-2022
  • (2021)Toward optimal mc/dc test case generationProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464841(505-516)Online publication date: 11-Jul-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing
April 2018
2327 pages
ISBN:9781450351911
DOI:10.1145/3167132
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 April 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. MC/DC
  2. SAT-based
  3. boolean expressions
  4. test data generation

Qualifiers

  • Research-article

Conference

SAC 2018
Sponsor:
SAC 2018: Symposium on Applied Computing
April 9 - 13, 2018
Pau, France

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Is MCDC Really Better? Lessons from Combining Tests and ProofsTests and Proofs10.1007/978-3-031-72044-4_2(25-44)Online publication date: 10-Sep-2024
  • (2022)Towards an Automatic Test Generation Method for Systems of Systems Based on Fault Injection and Model-Based Systems EngineeringApplied Sciences10.3390/app12221186312:22(11863)Online publication date: 21-Nov-2022
  • (2021)Toward optimal mc/dc test case generationProceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464841(505-516)Online publication date: 11-Jul-2021
  • (2021)MC/DC Test Cases Generation Based on BDDsDependable Software Engineering. Theories, Tools, and Applications10.1007/978-3-030-91265-9_10(178-197)Online publication date: 18-Nov-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media