Abstract
In this paper we discuss the advantages and limitations of a specification‐based software testing technique we call CEG‐BOR. There are two phases in this approach. First, informal software specifications are converted into cause‐effect graphs (CEG). Then, the Boolean OperatoR (BOR) strategy is applied to design and select test cases. The conversion of an informal specification into a CEG helps detect ambiguities and inconsistencies in the specification and sets the stage for design of test cases. The number of test cases needed to satisfy the BOR strategy grows linearly with the number of Boolean operators in CEG, and BOR testing guarantees detection of certain classes of Boolean operator faults. But, what makes the approach especially attractive is that the BOR based test suites appear to be very effective in detecting other fault types. We have empirically evaluated this broader aspect of the CEG‐BOR strategy on a simplified safety‐related real‐time control system, a set of N‐version programs, and on elements of a commercial data‐base system. In all cases, CEG‐BOR testing required fewer test cases than those generated for the applications without the use of CEG‐BOR. Furthermore, in all cases CEG‐BOR testing detected all faults that the original, and independently generated, application test‐suites did. In two instances CEG‐BOR testing uncovered additional faults. Our results indicate that the CEG‐BOR strategy is practical, scalable, and effective across diverse applications. We believe that it is a cost‐effective methodology for the development of systematic specification‐based software test‐suites.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Basili, V.R. and R.W. Selby (1987), “Comparing the Effectiveness of Software Testing Strategies,” IEEE Transactions on Software Engineering SE-13,12, 1278–1296.
Beizer, B. (1990), Software Testing Techniques, Second Edition, Van Nostrand Reinhold, New York, NY.
Bender, R.A. (1991), Requirements Based Testing CASE Tool, SoftTest, Bender and Associates, Larkspur, CA.
Boehm, B. (1989), Tutorial: Software Risk Management, IEEE Computer Society Press, Washington, DC.
Elmendorf, W.R. (1973), “Cause-Effect Graphs in Functional Testing,” TR-00.2487, IBM Systems Development Division, Poughkeepsie, NY.
Foreman, L. and S.H. Zweben (1993), “A Study of the Effectiveness of Control and Data Flow Testing Strategies,” Journal of Systems and Software 21,3, 215–228.
Frankl, P. and E.J. Weyuker (1988), “An Applicable Family of Data Flow Testing Criteria,” IEEE Transactions on Software Engineering 14,10, 1483–1498.
Hamlet, R. (1995), “Software Testing,” In Advances in Computers, M.V. Zelkovitz, Ed., Academic Press, San Diego, CA, pp. 1402–1411.
Howden, W.E. (1987), Functional Program Testing and Analysis, McGraw-Hill, New York, NY.
IEEE-ANSI (1989), IEEE Guide for the User of IEEE Standard Dictionary of Measures to Produce Reliable Software, IEEE Std 982.2-1988, IEEE Standards Board and ANSI, New York, NY.
Institute for Risk Research (1993), Generic Problem Competition: A Component of the International Symposium on Design and Review of Software Controlled Safety-Related Systems, Software Specification for the Generic Problem Competition, Ottawa, Ontario, Canada.
Konz, S. (1995), Work Design — Industrial Ergonomics, Fourth Edition, Publishing Horizons, Inc., Scotsdale, AZ.
Laski, J. (1989), “Testing in the Program Development Cycle,” Software Engineering Journal 4,2, 95–106.
Lyu, M. (1994), Handbook of Software Reliability Engineering, IEEE Computer Society Press and McGraw-Hill, New York, NY.
Musa, J.D., A. Iannino and K. Okumoto (1987), Software Reliability: Measurement, Prediction, Application, McGraw-Hill, New York, NY.
Myers, G.J. (1979), The Art of Software Testing, Wiley, New York, NY.
Nagle, P.M. and J.A. Skrivan (1982), “Software Reliability: Repetitive Run Experimentation and Modeling,” Bsc-40336, Boeing Inc., Seattle, WA.
Nursimulu, K. and R.L. Probert (1995), “Cause-Effect Graphing Analysis and Validation of Requirements,” In Proceedings of CASCON'95, IBM Canada Ltd. and National Research Council, Toronto, Ontario, Canada, p. 293.
Paradkar, A.M. (1996), “Specification Based Testing Using Cause-Effect Graphs,” PhD Dissertation, Department of Computer Science, North Carolina State University, Raleigh, NC.
Paradkar, A.M., I. Shields and J. Waters (1993), “The NCSU Solution to the Generic Problem Exercise: Boiler Control and Monitoring System,” Software Documentation for NCSU Solution to the Generic Problem Exercise, Department of Computer Science, North Carolina State University, Raleigh, NC.
Paradkar, A.M. and K.C. Tai (1995), “Test Generation for Boolean Expressions,” In Proceedings of International Symposium on Software Reliability Engineering'95, IEEE Computer Society Press, Los Alamitos, CA, pp. 106–115.
Paradkar, A.M., K.C. Tai and M.A. Vouk (1996), “Automatic Test Generation for Predicates,” IEEE Transactions on Reliability 45,4, 515–530.
Tai, K.C. (1996), “Theory of Fault-Based Predicate Testing for Computer Programs,” IEEE Transactions on Software Engineering 22,8, 552–562.
Tai, K.C., M.A. Vouk, A. Paradkar and P. Lu (1994), “Evaluation of a Predicate-Based Software Testing Strategy,” IBM Systems Journal 33,3, 445–457.
Vouk, M.A. and R.E. Coyle (1989), “BGG: A Testing Coverage Tool,” In Proceedings of 7th Annual Pacific Northwest Software Quality Conference, Lawrence and Craig, Inc., Portland, OR, pp. 212–233.
Vouk, M.A., M.L. Helsabeck, D.F. McAllister and K.C. Tai (1986a), “On Testing of Functionally Equivalent Components of Fault-Tolerant Software,” In Proceedings of Computer Software and Applications Conference (COMPSAC'86), IEEE Computer Society Press, Los Alamitos, CA, pp. 414–419.
Vouk, M.A., D.F. McAllister and K.C. Tai (1986b), “An Experimental Evaluation of the Effectiveness of Random Testing of Fault-Tolerant Software,” In Proceedings of Workshop on Software Testing, IEEE Computer Society Press, Los Alamitos, CA, pp. 74–81.
Vouk, M.A. and A.M. Paradkar (1993), “Design and Review of Software Controlled Safety-Related Systems: The NCSU Experience with the Generic Problem Exercise,” In Proceedings of the 1993 International Invitational Workshop on Design and Review of Software Controlled Safety-Related Systems, Institute for Risk Research, Ottawa, Ontario, Canada, pp. 209–222.
Weyuker, E.J. (1988), “An Empirical Study of the Complexity of Data Flow Testing,” In Proceedings of the Second Workshop on Software Testing, Verification, and Analysis, IEEE Computer Society Press, Los Alamitos, CA, pp. 188–195.
Weyuker, E.J., T. Goradia and A. Singh (1994), “Automatically Generating Test Data from a Boolean Specification,” IEEE Transactions on Software Engineering 20,5, 353–363.
White, L. (1995), “Software Testing and Verification,” In Encyclopedia of Microcomputers, Volume 16, A. Kent and J. Williams, Eds., Marcel Dekker Inc., New York, NY, pp. 185–241.
Yokoi, S. and M. Ohba (1992), “TCG: CEG-Based Tool and Its Experiments,” In Proceedings of the 13th Software Reliability Symposium, IEEE Computer Society Press, Los Alamitos, CA, pp. 41–49.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Paradkar, A., Tai, K. & Vouk, M. Specification‐based testing using cause‐effect graphs. Annals of Software Engineering 4, 133–157 (1997). https://doi.org/10.1023/A:1018979130614
Issue Date:
DOI: https://doi.org/10.1023/A:1018979130614