Skip to main content
SpringerLink
Log in

Test input generation from cause–effect graphs

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Cause–effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause–effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers’ method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause–effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause–effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause–effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers’ technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause–effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause–effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause–effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://github.com/denizkavzak/TOUCH

References

  • Alspaugh, T. A., Faulk, S. R., Britton, K. H., Parker, R. A., & Parnas, D. L. (1992). Software requirements for the A-7E aircraft. Naval Research Lab, Washington DC: Tech. rep.

    Google Scholar 

  • Atlee, J. M., & Buckley, M. A. (1996). A logic-model semantics for SCR software requirements. ACM SIGSOFT Software Engineering Notes, 21(3), 280–292.

    Article  Google Scholar 

  • Badhera, U., G.N., P., & Taruna, S. (2011). Fault based techniques for testing boolean expressions : A survey. International Journal of Computer Science & Engineering 3(1), 81–90.

  • C. A. S. T (2011) Rationale for Accepting Masking MC/DC in Certification Projects.

  • Chekam, T. T., Papadakis, M., Le Traon, Y., & Harman, M. (2017). An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption. In IEEE/ACM International Conference on Software Engineering (ICSE), pp. 597–608.

  • Chen, T., Lau, M., & Yu, Y. (1999). MUMCUT: A fault-based strategy for testing boolean specications. In Asia-Pacic Software Engineering Conference, p. 606.

  • Chen, T. Y., Grant, D. D., Lau, M. F., Ng, S. P., & Vasa, V. (2003). Beat:boolean expression fault-based test case generator. In Information Technology:Research and Education Conference, pp. 625–629.

  • Chen, T. Y., & Lau, M. F. (1997). Two test data selection strategies towards testing of boolean specifications. In Annual International Computer Software and Applications Conference (COMPSAC), pp. 608–611.

  • Chen, T. Y., & Lau, M. F. (2001). Test case selection strategies based on boolean specifications. Software Testing, Verification, Reliability, 11, 165–180.

    Article  Google Scholar 

  • Chen, T. Y., Lau, M. F., Sim, K. Y., & Sun, C. (2009). On detecting faults for boolean expressions. Software Quality Journal, 17(3), 245–261.

    Article  Google Scholar 

  • Chen, Z., Chen, T. Y., & Xu, B. (2011). A revisit of fault class hierarchies in general boolean specifications. ACM Transactions on Software Engineering and Methodology 20, 3, 13:1–13:11.

  • Chilenski, J., & Miller, S. (1994). Applicability of modified condition/decision coverage to software testing. Software Engineering Journal, 9(5), 193–200.

    Article  Google Scholar 

  • Chilenski, J. J. (2001). An investigation of three forms of the modified condition decision coverage (MCDC) criterion. DTIC Document: Tech. rep.

    Google Scholar 

  • de Oliveira Neto, F. G., Torkar, R., Feldt, R., Gren, L., Furia, C. A., & Huang, Z. (2019). Evolution of statistical analysis in empirical software engineering research: Current state and steps forward. Journal of Systems and Software, 156, 246–267.

    Article  Google Scholar 

  • Elmendorf, W. R. (1973). Cause-effect graphs in functional testing. IBM: Tech. rep.

    Google Scholar 

  • Falkowski, B. J. (2002). Spectral testing of digital circuits. VLSI Design, 14(1), 83–105.

    Article  MathSciNet  Google Scholar 

  • Foster, K. (1984). Sensitive test data for logic expressions. ACM SIGSOFT software engineering notes, 9(2), 120–125.

    Article  Google Scholar 

  • Gargantini, A., & Fraser, G. (2011). Generating minimal fault detecting test suites for general boolean specifications. Information and Software Technology, 53(11), 1263–1273.

    Article  Google Scholar 

  • Gephi. (2019). Gephi: The open graph viz platform. https://gephi.org/

  • GraphML. (2019) The graphml file format. https://graphml.graphdrawing.org/

  • Heitmeyer, C. L., & Bharadwaj, R. (2000). Applying the SCR requirements method to the light control case study. Journal of Universal Computer Science, 6(7), 650–678.

    Google Scholar 

  • Jin, Z. (1996). Deriving mode invariants from SCR specifications. In IEEE International Conference on Engineering of Complex Computer Systems (ICECCS, pp. 514–521.

  • Kaminski, G. K., & Ammann, P. (2009). Using logic criterion feasibility to reduce test set size while guaranteeing fault detection. In International Conference on Software Testing Verification and Validation (ICST), pp. 356–365.

  • Kapoor, K., & Bowen, J. P. (2007). Test conditions for fault classes in boolean specifications. ACM Transactions on Software Engineering and Methodology, 16(3), 10.

    Article  Google Scholar 

  • Kavzak Ufuktepe, D. (2016). Test case generation from cause effect graphs. Master’s thesis, Izmir Institute of Technology.

  • Kuhn, R. (1999). Fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology, 8(4), 411–424.

    Article  Google Scholar 

  • Lau, M. F., & Yu, Y. T. (2005). An extended fault class hierarchy for specification-based testing. ACM Transactions on Software Engineering and Methodology, 14(3), 247–276.

    Article  Google Scholar 

  • Mathur, P. A. (2008). Foundations of Software Testing, first ed. Pearson Publication.

  • Myers, G. J., Sandler, C., & Badgett, T. (2012). The Art of Software Testing, vol. 3.

  • Nursimulu, K., & Probert, R. L. (1995) Cause-effect graphing analysis and validation of requirements. In Conference of the Centre for Advanced Studies on Collaborative research (CASCON), IBM Press, p. 46.

  • O’Donnell, R. (2008). Some topics in analysis of boolean functions. Annual ACM symposium on Theory of computing (STOC), 569.

  • O’Donnell, R., & Austrin, P. (2012). Analysis of boolean functions: Notes from a series of lectures by ryan o’donnell. Workshop on Computational Complexity.

  • Offutt, J., Liu, S., Abdurazik, A., & Ammann, P. (2003). Generating test data from state-based specifications. Software testing, verification and reliability, 13(1), 25–53.

    Article  Google Scholar 

  • Papadakis, M., Delamaro, M., & Le Traon, Y. (2014). Mitigating the effects of equivalent mutants with mutant classification strategies. Science of Computer Programming, 95, 298–319.

    Article  Google Scholar 

  • Papadakis, M., Henard, C., Harman, M., Jia, Y., & Le Traon, Y. (2016). Threats to the validity of mutation-based test assessment. In ACM SIGSOFT International Symposium on Software Testing and Analysis ISSTA, pp. 354–365.

  • Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Le Traon, Y., & Harman, M. (2019). Mutation testing advances: An analysis and survey.

    Google Scholar 

  • Paradkar, A. (1994). On the experience of using cause-effect graphs for software specification and test generation. In Conference of the Centre for Advanced Studies on Collaborative Research (CASCON), IBM Press, p. 51.

  • Paradkar, A. (1995). A new solution to test generation for boolean expressions. In Conference of the Centre for Advanced Studies on Collaborative Research (CASCON), IBM Press, pp. 48

  • Paradkar, A., Tai, K. C., & Vouk, M. (1997). Specification-based testing using cause-effect graphs. Annals of Software Engineering, 4, 133–157.

    Article  Google Scholar 

  • Paradkar, A., Tai, K. C., & Vouk, M. A. (1996). Automatic test-generation for predicates. IEEE Transactions on Reliability, 45(4), 515–530.

    Article  Google Scholar 

  • Podgursky, B. (2013) https://github.com/bpodgursky/jbool_expressions/

  • Reed, I. S. (1973). Boolean Difference Calculus and Fault Finding. SIAM Journal on Applied Mathematics, 24(1), 134–143.

    Article  Google Scholar 

  • Singh, R. K., Chandra, P., & Singh, Y. (2006). An evaluation of boolean expression testing techniques. ACM SIGSOFT Software Engineering Notes, 31(5), 1–6.

    Article  Google Scholar 

  • Srivastava, P. R., Patel, P., & Hatrola, S. (2009). Cause effect graph to decision table generation. ACM SIGSOFT Software Engineering Notes, 34, 2.

    Google Scholar 

  • Sun, C. A., Zai, Y., & Liu, H. (2015). Evaluating and comparing fault-based testing strategies for general boolean specifications: A series of experiments. The Computer Journal, 58(5), 1199–1213.

    Article  Google Scholar 

  • Susskind, A. K. (1983). Testing by verifying walsh coefficients. IEEE Transactions on Computers, 32(2), 198–201.

    Article  Google Scholar 

  • Sziray, J. (2013) Evaluation of boolean graphs in software testing. In IEEE International Conference on Computational Cybernetics (ICCC), pp. 225–230.

  • Tai, K. C., Paradkar, A., Su, H. K., & Vouk, M. A. (1993) Fault-based test generation for cause-effect graphs. In Conference of the Centre for Advanced Studies on Collaborative Research: Software Engineering (CASCON), IBM Press, pp. 495–504.

  • Tsuchiya, T., & Kikuno, T. (2002). On fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology, 11(1), 58–62.

    Article  Google Scholar 

  • Vilkomir, S., Starov, O., & Bhambroo, R. (2013). Evaluation of t-wise approach for testing logical expressions in software. In IEEE International Conference on Software Testing, Verification and Validation Workshops, pp. 249–256.

  • Wang, Z., Chen, Z., Chen, T. Y., & Xu, B. (2012). Fault class prioritization in boolean expressions. In Annual ACM Symposium on Applied Computing, pp. 1191–1196.

  • Wang, Z., & Yu, M. (2018). BoolMuTest: A prototype tool for fault-based boolean-specification testing (p). In The International Conference on Software Engineering and Knowledge Engineering (SEKE), pp. 720–719.

  • Weyuker, E., Goradia, T., & Singh, A. (1994). Automatically generating test data from a boolean specification. IEEE Transactions on Software Engineering, 20(5), 353–363.

    Article  Google Scholar 

  • Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012). Experimentation in software engineering. Springer Science & Business Media.

  • Wolf, R. D. (2008). A brief introduction to fourier analysis on the boolean cube. Theory of Computing, Graduate Surveys, 1, 1–20.

    Google Scholar 

  • Yogi, N. (2009). Spectral Methods for Testing of Digital Circuits. PhD thesis, Auburn University.

  • Yogi, N., & Agrawal, V. (2006). High-level test generation for gate-level fault coverage. IEEE North Atlantic Test Workshop.

  • Yoo, S., & Harman, M. (2012). Regression testing minimization, selection and prioritization: a survey. Software Testing, Verification and Reliability, 22(2), 67–120.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Nevin Guler Dincer from the Statistics Department, Faculty of Science in Mugla Sitki Kocman University, for her help in the statistical testing. We also would like to thank Jaired Collins from the Electrical and Computer Science Department, College of Engineering in the University of Missouri-Columbia for his help in the proofreading of the manuscript.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Missouri-Columbia, Missouri, Columbia, USA

    Deniz Kavzak Ufuktepe

  2. Department of Computer Engineering, Izmir Institute of Technology, Izmir, Turkey

    Tolga Ayav & Fevzi Belli

  3. Department of Electrical Engineering and Information Technology, Paderborn University, Paderborn, Germany

    Fevzi Belli

Authors
  1. Deniz Kavzak Ufuktepe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tolga Ayav
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fevzi Belli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tolga Ayav.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kavzak Ufuktepe, D., Ayav, T. & Belli, F. Test input generation from cause–effect graphs. Software Qual J 29, 733–782 (2021). https://doi.org/10.1007/s11219-021-09560-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-021-09560-3

Keywords

Search

Navigation