skip to main content
10.1145/3018896.3036375acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccConference Proceedingsconference-collections
research-article

A comprehensive investigation of natural language processing techniques and tools to generate automated test cases

Published: 22 March 2017 Publication History

Abstract

Natural Language Processing (NLP) techniques show promising results to organize and identify desired information from the bulky raw data. As a result, NLP techniques are continuously getting researcher's attention to automate various software development activities like test cases generation. However, selection of right NLP techniques and tools to generate automated test cases is always challenging. Therefore, in this paper, we investigate the application of NLP techniques to generate test cases from preliminary requirements document. A Systematic Literature Review (SLR) has been conducted to identify 16 research works published during 2005-2014. Consequently, 6 NLP techniques and 18 tools have been identified. Furthermore, 4 test case generation approaches and 9 NLP algorithms have also been presented. The identified NLP techniques and tools are highly beneficial for the researchers and practitioners of the domain.

References

[1]
Harry M. Sneed, "Testing against natural language Requirements", Seventh International Conference on Quality Software, 2007 IEEE
[2]
Ravi Prakash Verma, Dr. (Prof.) Md. Rizwan Beg, "Generation of Test Cases from Software Requirements Using Natural Language Processing", 6th International Conference on Emerging Trends in Engineering and Technology, 2013 IEEE
[3]
Anurag Dwarakanath and Shubhashis Sengupta, "Litmus: Generation of Test Cases from Functional Requirements in Natural Language", Springer-Verlag 2012, pp. 58--69
[4]
hunhui Wang, Fabrizio Pastore, Arda Goknil, Lionel Briand, Zohaib Iqbal, "Automatic Generation of System Test Cases from Use Case Specifications", ISSTA, 2015/ACM
[5]
Edgar Sarmiento, Julio Cesar Sampaio do Prado Leite, Eduardo Almentero, "C&L: Generating Model Based Test Cases from Natural Language Requirements Descriptions", 2014 IEEE
[6]
Monalisha Khandai, Arup Abhinna Acharya, Durga Prasad Mohapatra, "A Novel Approach of Test Case Generation for Concurrent Systems Using UML Sequence Diagram", 2011 IEEE
[7]
Roaa Elghondakly, Sherin Moussa, Nagwa Badr, "Waterfall and Agile Requirements-based Model for Automated Test Cases Generation", Seventh International Conference on Intelligent Computing and Information Systems (ICICIS'15), 2015 IEEE
[8]
Christopher B. Harris, Ian G. Harris, "Generating Formal Hardware Verification Properties from Natural Language Documentation", 9th International Conference on Semantic Computing (IEEE ICSC 2015), 2015 IEEE
[9]
Matthais Schnelte, "Generating Test Cases for Timed Systems from Controlled Natural Language Specifications", Third International Conference on Secure Software Integration and Reliability Improvement, 2009 IEEE
[10]
Mehdi Malekzadeh, Raja Noor Ainon, "An Automatic Test Case Generator for Testing Safety-Critical Software Systems", 2010 IEEE
[11]
Dante Torres, Daniel Leitão, Flávia Barros, "Motorola SpecNL: a Hybrid System to Generate NL Descriptions from Test Case Specifications", Proceedings of (HIS'06), 2006.
[12]
Matthias Riebisch, Michael Hübner, "Traceability-Driven Model Refinement for Test Case Generation", 12th International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS'05), 2005 IEEE
[13]
Harry M. Sneed, Chris Verhoef, "Natural Language Requirement Specification for Web Service Testing", 2013.
[14]
Mariano Ceccato, Alessandro Marchetto, Leonardo Mariani, Cu D. Nguyen, Paolo Tonella, "Do Automatically Generated Test Cases Make Debugging Easier? An Experimental Assessment of Debugging Effectiveness and Efficiency", ACM Transactions on Software Engineering and Methodology, Vol. 25, No. 1, Article 5, Pub. Date: November 2015
[15]
Gustavo Carvalho, Diogo Falcão, Flávia Barros, Augusto Sampaio, Alexandre Mota, Leonardo Motta, Mark Blackburn, "Test Case Generation from Natural Language Requirements based on SCR Specifications", SAC'13 March 18-22, 2013, Coimbra, Portugal, ACM 2013
[16]
Avik Sinha, Stanley M. Sutton Jr., Amit Paradkar, "Text2Test: Automated Inspection of Natural Language Use Cases", Third International Conference on Software Testing, Verification and Validation, 2010 IEEE
[17]
H. Cunningham, D. Maynard, K. Bontcheva, V. Tablan, C. Ursu, M. Dimitrov, M. Dowman, N. Aswani, and I. Roberts, Developing Language Processing Components with GATE, University of Sheffield, 2005.
[18]
Bird, S., Klein, E., Loper, E., 2009. Natural Language Processing with Python. O'ReillyMedia
[19]
Ratnaparkhi, A. (1998). Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. thesis, University of Pennsylvania.
[20]
Sleator, D.D.K., Temperley, D.: Parsing English with a Link Grammar. In: Third parsing Workshop (1993)
[21]
ADL project. Assertion Definition Language project website. 2000. Available in: http://adl.opengroup.org/index.html. Accessed in March 06, 2006.
[22]
Sneed, H.: "Testing against natural language Requirements", 7th IEEE International Conference on Software Quality (QSIC2007), Portland, Oct. 2007, p. 380
[23]
G. Fraser and A. Arcuri. 2011. Evolutionary generation of whole test suites. In Proceedings of the 11th International Conference on Quality Software (QSIC'11). 31--40.
[24]
C. Pacheco and M. D. Ernst. 2007. Randoop: Feedback-directed random testing for Java. In Companion to the 22nd ACMSIGPLAN Conference on Object-Oriented Programming Systems and Applications Companion (OOPSLA'07). ACM Press, New York, 815--816.
[25]
M. Blackburn, R. Busser, and J. Fontaine. Automatic Generation of Test Vectors for SCR-style Specications. In Proceedings of ACSAC, 1997.
[26]
J. Allen. Natural Language Understanding. Benjamin/Cummings, 1995
[27]
Kitchenham, B., 2004. Procedures for Performing Systematic Reviews. Keele University TR/SE-0401/NICTA, TechnicalReport0400011T.
[28]
Marie Meteer, Bensiin Borukhov, Michael Crivaro, Michael Shafir, Attapol Thamrongrattanarit "MedLingMap: A growing resource mapping the Bio-Medical NLP field", Proceedings of the 2012 Workshop on Biomedical Natural Language Processing (BioNLP 2012), pages 140--145, Montreal, Canada, June 8, 2012
[29]
Ashfa Umber, Imran Sarwar Bajwa, M. Asif Naeem, "NL-Based Automated Software Requirements Elicitation and Specification", ACC 2011 Proceedings, Part II, pp 30--39
[30]
Deva Kumar Deeptimahanti, Muhammad Ali Babar, "An Automated Tool for Generating UML Models from Natural Language Requirements", International Conference on Automated Software Engineering, IEEE/ACM, pp. 680--682
[31]
ACM, Last Accessed August 2016. http://dl.acm.org/
[32]
IEEE Scientific database, Last Accessed, August 2016. http://ieeexplore.ieee.org
[33]
Springer, Last Accessed August 2016. http://link.springer.com

Cited By

View all
  • (2024)Automation of Test Skeletons Within Test-Driven Development Projects2024 36th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET62301.2024.10663016(1-10)Online publication date: 29-Jul-2024
  • (2024)Natural Language Processing-Based Software Testing: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.340775312(79383-79400)Online publication date: 2024
  • (2024)User Story Based Automated Test Case Generation Using NLPComputational Intelligence in Data Science10.1007/978-3-031-69982-5_12(156-166)Online publication date: 30-Aug-2024
  • Show More Cited By

Index Terms

  1. A comprehensive investigation of natural language processing techniques and tools to generate automated test cases

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      ICC '17: Proceedings of the Second International Conference on Internet of things, Data and Cloud Computing
      March 2017
      1349 pages
      ISBN:9781450347747
      DOI:10.1145/3018896
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 March 2017

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. NLP
      2. SLR
      3. nature language processing
      4. test case generation
      5. test cases
      6. text mining

      Qualifiers

      • Research-article

      Conference

      ICC '17

      Acceptance Rates

      ICC '17 Paper Acceptance Rate 213 of 590 submissions, 36%;
      Overall Acceptance Rate 213 of 590 submissions, 36%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)62
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Automation of Test Skeletons Within Test-Driven Development Projects2024 36th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET62301.2024.10663016(1-10)Online publication date: 29-Jul-2024
      • (2024)Natural Language Processing-Based Software Testing: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.340775312(79383-79400)Online publication date: 2024
      • (2024)User Story Based Automated Test Case Generation Using NLPComputational Intelligence in Data Science10.1007/978-3-031-69982-5_12(156-166)Online publication date: 30-Aug-2024
      • (2024)Towards an Intelligent Test Case Generation Framework Using LLMs and Prompt EngineeringAdvances in Smart Medical, IoT & Artificial Intelligence10.1007/978-3-031-66854-8_3(24-31)Online publication date: 1-Sep-2024
      • (2024)Visualizing Software Test Requirements Using NLP and HITL ApproachInformation Management and Big Data10.1007/978-3-031-63616-5_22(288-298)Online publication date: 29-Jun-2024
      • (2023)SDLC Phases of a Mobile ApplicationDesigning and Developing Innovative Mobile Applications10.4018/978-1-6684-8582-8.ch013(232-249)Online publication date: 30-Jun-2023
      • (2023)A Decade of Intelligent Software Testing Research: A Bibliometric AnalysisElectronics10.3390/electronics1209210912:9(2109)Online publication date: 5-May-2023
      • (2023)SelectNLTest - Selection and natural language rewriting of test cases generated by the DRL-MOBTEST toolProceedings of the 8th Brazilian Symposium on Systematic and Automated Software Testing10.1145/3624032.3624043(77-85)Online publication date: 25-Sep-2023
      • (2023)Artificial Intelligence Applied to Software Testing: A Tertiary StudyACM Computing Surveys10.1145/361637256:3(1-38)Online publication date: 6-Oct-2023
      • (2023)Test case generation and history data analysis during optimization in regression testing: An NLP studyCogent Engineering10.1080/23311916.2023.227649510:2Online publication date: 12-Nov-2023
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media