Skip to main content
SpringerLink
Log in

Test Case Generation as an AI Planning Problem

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

While Artificial Intelligence techniques have been applied to a variety of software engineering applications, the area of automated software testing remains largely unexplored. Yet, test cases for certain types of systems (e.g., those with command language interfaces and transaction based systems) are similar to plans. We have exploited this similarity by constructing an automated test case generator with an AI planning system at its core. We compared the functionality and output of two systems, one based on Software Engineering techniques and the other on planning, for a real application: the StorageTek robot tape library command language. From this, we showed that AI planning is a viable technique for test case generation and that the two approaches are complementary in their capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Charles Anderson, Anneliese von Mayrhauser, and Rick Mraz. “On the Use of Neural Networks to Guide Software Testing Activities”, Procs. International Test Conference, Oct. 1995, Washington, DC.

  • John S. Anderson. Automating Requirements Engineering Using Artificial Intelligence Techniques. PhD thesis, Dept. of Computer and Information Science, University of Oregon, Dec. 1993.

  • Anthony Barrett, Keith Golden, Scott Penberthy, and Daniel Weld. UCPOP User's Manual. Dept of Computer Science and Engineering, University of Washington, Seattle, WA, October 1993. TR 93-09-06.

    Google Scholar 

  • J. Bauer and A. Finger. “Test Plan Generation Using Formal Grammars”, Procs. Fourth International Conference on Software Engineering, 1979, pp. 425-432.

  • Franco Bazzichi and Ippolito Spadafora. “An Automatic Generator for Compiler Testing,” IEEE Transactions on Software Engineering, 1982:8(4), pp.343-353.

    Google Scholar 

  • Ted J. Biggerstaff and Alan J. Perlis. Software Reusability: Volume I, Concepts and Models, ACM Press, Fronier Series, 1989.

  • Grady Booch, Object Oriented Design with Applications, ”Benjamin/Cummings”, 1991.

  • A. Celentano, S. Crespi Reghizzi, P. Della Vigna, C. Ghezzi, G. Gramata and F. Savoretti. “Compiler Testing using a Sentence Generator,” Software-Practice and Experience, 1980:10, pp. 987-918.

    Google Scholar 

  • John J. Chilenski and Philip H. Newcomb. “Formal Specification Tools for Test Coverage Analysis”, Procs. Ninth Knowledge-Based Software Engineering Conference, September 1994, Monterey, CA, pp. 59-68.

  • Paul R. Cohen and Edward A. Feigenbaum. Handbook of Artificial Intelligence, volume 3, chapter Planning and Problem Solving, pages 513-562. William Kaufmann, Inc., Los Angeles, 1982.

    Google Scholar 

  • A.G. Duncan and J.S. Hutchison, “Using Attributed Grammars to Test Designs and Implementations,” Proceedings of the Fifth International Conference on Software Engineering, 1981, pp. 170-177.

  • Tsum S. Chow. “Testing Software Design Modeled by Finite State Machines,” Proceedings of the First COMPSAC, 1977, pp. 58-64.

  • W. Deason, D. Brown, K.-H. Chang, and J. Cross. “Rule-Based Software Test Data Generator”, IEEE Transactions on Knowledge and Data Engineering, 3(1) March 1991, pp. 108-117.

    Google Scholar 

  • Stephen Fickas and John Anderson. A proposed perspective shift: Viewing specification design as a planning problem. Department of Computer and Information Science CIS-TR-88-15, University of Oregon, Eugene, OR, November 1988.

    Google Scholar 

  • Stephen Fickas and B. Robert Helm. “Knowledge Representation and Reasoning in the Design of Composite Systems”, IEEE Transactions on Software Engineering, SE-18(6), June 1992, pp. 470-482.

    Google Scholar 

  • Tom Figliulo, Anneliese von Mayrhauser, and Richard Karcich. “Experiences with Automated System Testing and SLEUTH”, Procs. IEEE Aerospace Applications Conference 1996, February 1996.

  • S. Fujiwara, G. von Bochman, F. Khendek, M. Amalou, and A. Ghedamsi. “Test Selection Based on Finite State Models”, IEEE Transactions on Software Engineering SE-17, no. 10(June 1991), pp. 591-603.

    Google Scholar 

  • J. B. Goodenough and S. L. Gerhart. “Toward a Theory of Test Data Selection”, IEEE Transactions on Software Engineering, SE-1(2), June 1975, pp. 156-173.

    Google Scholar 

  • Dick Hamlet and Ross Taylor. “Partition Testing Does not inspire Confidence”, IEEE Transactions on Software Engineering, SE-16(12), Dec. 1990, pp. 1402-1411.

    Google Scholar 

  • James W. Hooper and Rowena O. Chester. Software Reuse: Guidelines and Methods, Plenum Publishers, 1991.

  • Karen Huff and Victor Lesser. A plan-based intelligent assistant that supports the software development process. In ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, Nov. 1988.

  • Karen Huff. Software adaptation. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse, pages 63-66, Stanford University, March 1992.

  • D. C. Ince. “The Automatic Generation of Test Data”, Computer Journal, vol.30(1), 1987, pp. 63-69.

    Google Scholar 

  • P. Maurer. “Generating Test Data with Enhanced Context-Free Grammars”, IEEE Software, July 1990, pp. 50-55.

  • Glenford J. Myers. The Art of Software Testing, Wiley Series in Business Data Processing. John Wiley and Sons, 1979.

  • Thomas J. Ostrand and Marc J. Balcer. “The Category-Partition Method for Specifying and Generating Functional Tests”, Communications of the ACM, 31(6), June 1988, pp. 676-686.

    Google Scholar 

  • J.S. Penberthy and D. Weld. “UCPOP: A sound, complete, partial order planner for ADL”, In Proceedings Third International Conference on Principles of Knowledge Representation and Reasoning, October 1992, pp. 103-114.

  • P. Purdom. “A Sentence Generator for Testing Parsers”, BIT, 12(3), 1972, pp. 366-375.

    Google Scholar 

  • Debra J. Richardson, Owen O'Malley, and Cindy Tittle. “Approaches to Specification-Based Testing”, Procs. ACM Third Symposium on Software Testing, Analysis, and Verification (TAV3), December 1993, pp. 86-96.

  • Robert S. Rist. Plan Identification and Re-use in Programs. In Working Notes of AAAI-92 Spring Symposium on Computational Considerations in Supporting Incremental Modification and Reuse, pages 67-72, Stanford University, March 1992.

  • Kenneth S. Rubin and Adele Goldberg. “Object Behavior Analysis,” Communications of the ACM, 35(9), September 1992, pp. 48-62.

    Google Scholar 

  • Raghavan Srinivasan and Adele E. Howe. Comparison of Methods for Improving Search Efficiency in a Partial-Order Planner. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 1620-1626, Montreal, Canada, August 1995.

  • Storage Tek, Storage Tek 4400 Operator's Guide, Host Software Component (VM) Rel 1.2.0, Storage Tek, 1992.

  • Gerald A. Sussman. A computational model of skill acquisition. Technical Report Memo no. AI-TR-297, MIT AI Lab, 1973.

  • Markos Z. Tsoulakas, Joe W. Duran, and Simeon C. Ntafos. “On Some Reliability Estimation Problems in Random and Partition Testing”, IEEE Transactions on Software Engineering, 19(7), July 1993, pp. 687-697.

    Google Scholar 

  • Anneliese von Mayrhauser and Steward Crawford-Hines, “Automated Testing Support for a Robot Tape Library,” Proceedings of the Fourth International Software Reliability Engineering Conference, November 1993, pp. 6-14.

  • Anneliese von Mayrhauser, Richard T. Mraz, and Jeff Walls. “Domain Based Regression Testing,” Proceedings of the International Conference on Software Maintenance, Sept 1994, p. 26-35.

  • Anneliese von Mayrhauser, Richard Mraz, Jeff Walls, and Pete Ocken. “Domain Based Testing: Increasing Test Case Reuse,” Proc. of the International Conference on Computer Design, October 1994, p. 484-491.

  • Anneliese von Mayrhauser, Jeff Walls, and Richard Mraz, “Testing Applications Using Domain Based Testing and Sleuth,” Proceedings of the Fifth International Software Reliability Engineering Conference, November 1994, p. 206-215.

  • Anneliese von Mayrhauser, Jeff Walls, and Richard Marz. “Sleuth: A Domain Based Testing Tool,” Proc. of the International Test Conference, October 1994, p. 840-849.

  • Elaine J. Weyuker and Bingchiang Jeng. “Analyzing Partition Testing Strategies”, IEEE Transactions on Software Engineering, 17(7), July 1991, pp. 703-711.

    Google Scholar 

  • Steven J. Zeil and Christian Wild. “A Knowledge Base for Software Test Refinement”, Technical Report TR-93-14, Old Dominion University, Norfolk VA.

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, Colorado State University, Fort Collins, CO, 80523

    Adele E. Howe & Anneliese von Mayrhauser

  2. HQ USAFA/DFCS, U.S. Air Force Academy, 2354 Fairchild Hall, Suite 6K41, CO, 80840

    Richard T. Mraz

Authors
  1. Adele E. Howe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anneliese von Mayrhauser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard T. Mraz
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Howe, A.E., Mayrhauser, A.v. & Mraz, R.T. Test Case Generation as an AI Planning Problem. Automated Software Engineering 4, 77–106 (1997). https://doi.org/10.1023/A:1008607721339

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008607721339

Search

Navigation