Abstract
Design by contract is a software development methodology that uses contracts for defining interfaces among interacting components of a software system. Graph transformation system is used to specify the behavioral aspects of software components by defining the pre- and post-conditions of methods as contracts. In this paper, we focus on testing executable software models specified by a graph transformation system. A set of model-specific coverage criteria and a cost-aware search-based test generation approach are introduced. To evaluate the effectiveness of the proposed coverage criteria and the test generation approach, a type of mutation analysis is presented at the model level. Furthermore, a couple of fault-detection methods are used to assess the quality of the generated tests in the model-level mutation analysis. The proposed approach is implemented in GROOVE, a toolset for model checking graph transformation systems. The empirical results based on some well-known case studies demonstrate the efficiency and scalability of each proposed coverage criterion and testing approach. The comparison of the proposed test generation approach with state-of-the-art techniques indicates a significant improvement in terms of fault-detection capability and testing costs.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Liu S (2016) Validating formal specifications using testing-based specification animation. In: Proceedings of the 4th FME workshop on formal methods in software engineering. ACM
Belli F et al (2016) Model-based mutation testing—approach and case studies. Sci Comput Program 120:25–48
Sargent RG (2013) Verification and validation of simulation models. J Simul 7(1):12–24
Granda M et al (2017) Effectiveness assessment of an early testing technique using model-level mutants. In: Proceedings of the 21st international conference on evaluation and assessment in software engineering. ACM
Wong PY et al (2015) Testing abstract behavioral specifications. Int J Softw Tools Technol Transf 17(1):107–119
Kemmerer RA (1985) Testing formal specifications to detect design errors. IEEE Trans Softw Eng 1:32–43
González CA et al (2018) Enabling model testing of cyber-physical systems. In: Proceedings of the 21th ACM/IEEE international conference on model driven engineering languages and systems
Araujo W, Briand LC, Labiche Y (2014) On the effectiveness of contracts as test oracles in the detection and diagnosis of functional faults in concurrent object-oriented software. IEEE Trans Softw Eng 40(10):971–992
Meyer B (1988) Object-oriented software construction, vol 2. Prentice Hall, New York
Ehrig H, Ehrig K (2006) Overview of formal concepts for model transformations based on typed attributed graph transformation. Electron Notes Theor Comput Sci 152:3–22
Heckel R (2006) Graph transformation in a nutshell. Electron Notes Theor Comput Sci 148(1):187–198
Taentzer G (2003) AGG: a graph transformation environment for modeling and validation of software. In: International workshop on applications of graph transformations with industrial relevance. Springer
König B et al (2018) A tutorial on graph transformation. In: Graph transformation, specifications, and nets. Springer, pp 83–104
Machado R, Ribeiro L, Heckel R (2015) Rule-based transformation of graph rewriting rules: towards higher-order graph grammars. Theor Comput Sci 594:1–23
Kalaee A, Rafe V (2018) Model-based test suite generation for graph transformation system using model simulation and search-based techniques. Information and Software Technology
Baier C, Katoen J-P (2008) Principles of model checking. MIT Press, Cambridge
Xu D et al (2016) Automated integration testing using logical contracts. IEEE Trans Reliab 65(3):1205–1222
Rafe V, Rahmani M, Rashidi K (2013) A survey on coping with the state space explosion problem in model checking. Int Res J Appl Basic Sci 4(6):1379–1384
Enoiu EP et al (2016) Automated test generation using model checking: an industrial evaluation. Int J Softw Tools Technol Transf 18(3):335–353
Villani E et al (2019) Integrating model checking and model based testing for industrial software development. Comput Ind 104:88–102
Fraser G, Wotawa F, Ammann P (2009) Issues in using model checkers for test case generation. J Syst Softw 82(9):1403–1418
Ferrer J et al (2015) Search based algorithms for test sequence generation in functional testing. Inf Softw Technol 58:419–432
Khan TA, Runge O, Heckel R (2012) Testing against visual contracts: model-based coverage. In: International conference on graph transformation. Springer
Runge O, Khan TA, Heckel R (2013) Test case generation using visual contracts. In: Electronic Communications of the EASST, vol 58
de Bruijn V (2013) Model-based testing with graph grammars
Rensink A (2003) The GROOVE simulator: a tool for state space generation. In: International workshop on applications of graph transformations with industrial relevance. Springer
Rozenberg G (1997) Handbook of graph grammars and computing by graph transformation, vol 1. World Scientific, Singapore
Kastenberg H, Rensink A (2006) Model checking dynamic states in GROOVE. In: International SPIN workshop on model checking of software. Springer
Varró D, Balogh A (2007) The model transformation language of the VIATRA2 framework. Sci Comput Program 68(3):214–234
Heckel R, Khan TA, Machado R (2011) Towards test coverage criteria for visual contracts. In: Electronic Communications of the EASST, vol 41
Koopman P, Achten P, Plasmeijer R (2008) Testing and validating the quality of specifications. In: 2008 IEEE international conference on software testing verification and validation workshop. IEEE
Jard C, Bochmann GV (1983) An approach to testing specifications. J Syst Softw 3(4):315–323
Anand S et al (2013) An orchestrated survey of methodologies for automated software test case generation. J Syst Softw 86(8):1978–2001
Fraser G, Wotawa F, Ammann PE (2009) Testing with model checkers: a survey. Softw Test Verif Reliab 19(3):215–261
Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST). IEEE
Khari M, Kumar P (2017) An extensive evaluation of search-based software testing: a review. Soft Comput 1–14
Fraser G, Arcuri A (2013) Whole test suite generation. IEEE Trans Softw Eng 39(2):276–291
Heckel R, Mariani L (2004) Component integration testing by graph transformations. In: International conference on computer science, software engineering, information technology, e-Business, and applications, Cairo. Citeseer
Gönczy L, Heckel R, Varró D (2007) Model-based testing of service infrastructure components. In: Testing of software and communicating systems. Springer. pp 155–170
Legay A et al (2017) On featured transition systems. In: International conference on current trends in theory and practice of informatics. Springer
Harman M et al (2010) Optimizing for the number of tests generated in search based test data generation with an application to the oracle cost problem. In: 2010 third international conference on software testing, verification, and validation workshops (ICSTW). IEEE
Arqub OA, Abo-Hammour Z (2014) Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm. Inf Sci 279:396–415
Grechanik M, Devanla G (2016) Mutation integration testing. In: 2016 IEEE international conference on software quality, reliability and security (QRS). IEEE
Offutt AJ (1992) Investigations of the software testing coupling effect. ACM Trans Softw Eng Methodol (TOSEM) 1(1):5–20
Ma Y-S, Offutt J, Kwon Y-R (2006) MuJava: a mutation system for Java. In: Proceedings of the 28th international conference on Software engineering. ACM
Schmidt A, Varró D (2003) CheckVML: a tool for model checking visual modeling languages. In: International conference on the unified modeling language. Springer
Engels G, Güldali B, Lohmann M (2006) Towards model-driven unit testing. In: International conference on model driven engineering languages and systems. Springer
Lobo F, Lima CF, Michalewicz Z (2007) Parameter setting in evolutionary algorithms, vol 54. Springer, Berlin
Lobo FG, Lima CF (2007) Adaptive population sizing schemes in genetic algorithms. In: Parameter setting in evolutionary algorithms. Springer, pp 185–204
Arqub OA (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm-Volterra integrodifferential equations. Neural Comput Appl 28(7):1591–1610
Arqub OA et al (2016) Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput 20(8):3283–3302
Arqub OA et al (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206
Malburg J, Fraser G (2011) Combining search-based and constraint-based testing. In: Proceedings of the 2011 26th IEEE/ACM international conference on automated software engineering. IEEE Computer Society
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Algorithm 4 shows how coverage goals concerning the introduced data-dependencies in a given test case are detected in GROOVE toolset. It receives a test sequence as input and keeps track of data-flow along the execution path. Lines 1–3 of the algorithm initializes needed data structures including four empty sets to maintain detected coverage goals and put the SUC to the start state. Then each transformation step of the test sequence is applied to the system state and experienced data-flow, i.e. def-use, are recorded through the while loop (Lines 4–26). In each transformation step, the applied rule was registered for created objects (def step, Lines 5–8) to detect the source of the data object in the use step. Next, all data-dependencies experienced by the applied transformation step extracted as Create_Read (Lines 9–13), Create_Delete (Lines 14–18), Create_Update (Lines 19–23), and Delete_NAC (Lines 24 and 25). To specify the Delete_NAC relations, as shown in Algorithm 5, we should check for each transition t how the elements deleted in the previous transitions enable transition t through the provision of its NAC conditions.
Algorithm 6 shows how we calculate the number of covered test objectives by all test cases of a test suite as a fitness of the test suite. In this algorithm initially, we have four empty sets (CreatRead, CreateDelete, CreateUpdate, DeleteNAC) to save test goals covered by the test suite (Line 1). Then, data-dependencies covered by each test case are extracted by Algorithm 1 (Line 3) and added to the corresponding sets of the test suite (Lines 4–7). Finally, the fitness value is calculated as a sum of the size of all introduced sets which selected as coverage criteria (Line 8, in this algorithm all of the introduced data-flow coverage criteria are considered).
Algorithm 7 is designed for generating a random test path. This algorithm launches in the start state and initializes a new empty test path (Lines 1 and 2), then selects randomly an outgoing transition from the current state and goes to the next state through the selected transition (Lines 4–6). The number of the selected transition added to the encoded test path (Line 7).
Rights and permissions
About this article
Cite this article
Bahrampour, A., Rafe, V. Using search-based techniques for testing executable software models specified through graph transformations. Int. J. Mach. Learn. & Cyber. 11, 2743–2770 (2020). https://doi.org/10.1007/s13042-020-01149-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-020-01149-9