Keywords

1 Introduction

Software migration is a well-established method for transferring software systems into new environments without changing their functionality. For example, in legacy migration, a technological obsolete system that is still valuable for ongoing business is migrated into a new environment [10]. Another use case is when a system is migrated to another platform, thus enabling a multi-platform provisioning (a common use case in mobile application development nowadays).

Software testing plays an important role in software migration as it verifies whether the main requirement, i.e., the preservation of the functionality, is fulfilled. Existing test cases, are important assets and their reuse can be beneficial, not just from economical perspective, as the expensive and time consuming test design is avoided [40], but also from practical perspective: the existing test cases contain valuable information about the functionality of the source system. So similarly, as software migration is established to reuse existing systems, we want to reuse test cases as well.

However, in many migration scenarios a direct reuse of test cases might be impossible due to system changes as well as the differences in the source and the target environments. As test cases are coupled with the system they are testing, the system changes need to be detected, understood and then reflected on the test cases to facilitate reuse. In other words, the test cases need to be co-migrated. But, the size of the test case set which sometimes is measured in tens of thousands of test cases, the missing conformity in the test case structure, make the migration of test cases a quite challenging task [28]. Hence, first of all, a new migration method should be able to deal with the structural complexity of the test cases. Then, it should also provide an easy way to restructure the test cases and reflect the relevant system changes. Last but not least, the migrated test cases should be appropriate for the target environment, i.e., the test cases have to be re-designed for the target environment.

The Model-Driven Engineering (MDE) software development methodology has been established to deal with the increasing complexity of the today’s software systems by focusing on creating and exploiting domain models. The Model-Driven Architecture (MDA) initiative, proposed by Object Management Group (OMG), puts the MDE idea in action, and clearly separates the business logic from implementation by defining software models at different level of abstraction. The general idea is to distinguish between platform-independent models (PIMs) and platform-specific models (PSMs).

Software migration approaches that follow the MDA principles are known as model-driven software migration (MDSD) approaches [20]. Generally speaking, software migration can be seen as a transformation of the system which is done by enacting software transformation method, which is an instance of the well-known horseshoe model [29]. Therefrom, software migration is some kind of software reengineering, which according to [13] is defined as “examination and alteration of a subject system to reconstitute it in a new form and the subsequent implementation of the new form”. In general, software reengineering consists of three consecutive phases: Reverse Engineering, Restructuring and Forward Engineering.

The MDA approach is also suitable for the test domain [22]. Software testing which relies on the MDA principles is known as model-driven testing [16, 23, 31]. Similarly to software migration, test case migration can be seen as a transformation of test cases implemented for a specific source technology to test cases implemented for a specific target technology.

Following the idea of model-driven software migration, we propose a reengineering horseshoe model for the domain of test cases. It contains the basic reengineering activities, Reverse Engineering, Restructuring, and Forward Engineering and artifacts on different levels of abstraction specific for software testing. Regarding the level of abstractions, it distinguishes between three levels of abstraction: System Layer, Platform-Independent Layer, and Platform-Specific Layer. Relying on the benefits from MDA and software reengineering, the main aim of the proposed reengineering horseshoe model is to provide (semi-)automated migration of test cases which ideally results in a high-quality migrated test case set, appropriate for the target environment. Only then one can be sure that the execution of the migrated test cases is a clear indicator for the successfulness of the system migration.

In this paper, we also present a case study from an industrial project in which a migration of an existing modeling framework from Java to C# was performed. Using the activities and the artifacts proposed in the reengineering model, we firstly instantiated a model-driven test case migration method, then we developed appropriate tooling that was supporting the migration method, and at the end the method was enacted. In total, 4000 test cases were migrated from jUnit [3] to MSUnit [4], most of them completely automatic.

The structure of the rest of the paper is as follows: In Sect. 2, we give an overview of the fundamentals important for our work. Then, in Sect. 3, we present the test case reengineering horseshoe model. Section 4 discusses the case study where our reengineering model was applied. In Sect. 5 we discuss the related work and at the end, Sect. 6 concludes the work and gives an outlook on future work.

2 Background

Model-Driven Engineering (MDE) has been established to deal with the increasing complexity of development, maintenance and evolution of the nowadays software systems. In order to achieve this, it relies on models and model transformations [11]. Model-Driven Architecture (MDA), proposed by Object Management Group (OMG), defines several software models on different level of abstraction, thus clearly separating the business complexity from the implementation details [37]. MDA defines three levels of abstraction: Computational-Independent Model (CIM), which focuses on the context and the requirements of the system, Platform-Independent Model (PIM), which main focus is on the operational capabilities of the system without consideration of a specific technology and Platform-Specific Model (PSM), which focuses on a specific platform.

Software migration is a well-established method for transferring software systems into new environments without changing their functionality. In case when MDA principles are followed, these migration approaches are known as model-driven software migration (MDSD) [20]. Software migration is some kind of software reengineering, which according to [13] is defined as “examination and alteration of a subject system to reconstitute it in a new form and the subsequent implementation of the new form”. In general, software reengineering consists of three consecutive phases: Reverse Engineering, Restructuring and Forward Engineering. Reverse Engineering, according to [13], is the process of analyzing a subject system to create representations of the system in another form or on a higher level of abstraction. Forward Engineering is defined as “the traditional process of moving from high-level abstractions and logical, implementation-independent designs to the physical implementation of a system” [13]. Restructuring, as defined by [13], is “the transformation from one representation form to another at the same relative abstraction level, while preserving the subject systems external behavior (functionality and semantics)”.

Model-Based Testing (MBT) [38] is a software testing methodology that uses (semi-)formal models which encode the expected behavior of the system under test to derive test artifacts like test cases, test data or test configuration. Model-Driven Testing (MDT) [16, 23] is a type of model-based testing which follows the Model Driven Engineering principles, i.e., the test cases are automatically generated from a test models using model transformations.

UML Testing Profile [36] is a language standardized by OMG which supports testing on model level. It can be divided into four main parts: Test Architecture, Test Behavior, Test Data, and Test Management. Test Architecture is used for specification of the structural aspects of the test environment and the corresponding test configuration. Test Behavior specifies the stimulation and observation of the SUT by using any kind of UML behavior diagram. Using Test Data one can specify the pre- and post-conditions as well as the input and expected output of the SUT. Last but not least, using Test Management one can manage for example, the test planing, scheduling or execution of test specifications.

Test Description Language (TDL) [18] is a testing language standardized by the European Telecommunications Standards Institute (ETSI)Footnote 1 bridges the gap between high-level test purpose specifications and executable test cases [41]. The language is scenario-based and it can be used for design, documentation, and representation of formalized test descriptions. The main ingredients of the language are Test Data, Test Configuration, Test Behavior, Test Objectives and Time. Test Data specifies the elements needed to express data sets and data instances which are used in test descriptions. Then, Test Configuration typed components and gates and the connections among the gates. Test Behavior defines the expected behavior. Using Test Objectives one can defines the objectives of the testing by attaching them to a behavior or to a whole test description.

Testing and Test Control Notation version 3 (TTCN-3) [17] is a testing language for test specification and implementation standardized by the European Telecommunications Standards Institute (ETSI). It supports creation of abstract test specifications which can be executed by providing additional implementation components, such as an SUT adapter.

xUnit [33] is a family of unit-testing frameworks that share common characteristics. Probably the most popular frameworks part of this family are jUnit [3] and nUnit [5] used for testing Java for C# software systems receptively. MSUnit is a Microsoft’s testing framework for unit testing integrated in Visual Studio which has a similar, but still a little bit different structure compared to the xUnit-family frameworks.

3 The Test Case Reengineering Horseshoe Model

In this section, we present the test case reengineering horseshoe model [29] for migration of test cases which is defined by following the idea of model-driven software migration [19,20,21] (Fig. 1).

It comprises the reengineering activities Reverse engineering, Restructuring, and Forward Engineering [13] and the corresponding artifacts in terms of textual artifacts and models on different levels of abstraction. Here, we present the conceptual method on model-driven migration of test cases and in the next section we come up with concrete examples from the case study that we have performed.

Figure 1.
figure 1

The test case reengineering horseshoe model

Full size image

3.1 Artifacts

Artifacts are constituting parts of each transformation method. With regards to the level of abstraction, it can be distinguished between three layers of abstraction: System Layer, Platform-Specific Layer, and Platform-Independent Level.

System Layer

On the System Layer, textual artifacts representing test code and models of the test code are placed. Regarding the textual artifacts, this is either the original test code or the migrated test code. Similarly, regarding the models it is either the model of the original test code or the model of the migrated test code.

Test Code (Original/Migrated): The test code can be either the test cases, implemented in a specific testing framework, e.g. jUnit [3] or MSUnit [4], test configuration scripts or manually implemented additional test framework. If the test cases are dependent on these artifacts, e.g., by using specific assert functions from the additional test framework, they also have to be considered and eventually migrated along with the test cases.

Model of the Test Code (Original/Target): The Model of the Test Code represents the test code in a form of Abstract Syntax Tree [9] of the appropriate language of the original or the target environment.

Platform-Specific Layer

The platform-specific layer is a higher level of abstraction compared to the system layer. Here, technology-specific concepts are used to represent the test cases for both the source and the target environment.

Model of Executable Tests (Original/Target): This model is considered as a platform-specific as it represents the executable test cases by using testing concepts which are specific for a particular testing framework, i.e., by using meta-models of jUnit [3] or MSUnit [4] to model the tests.

Platform-Independent Layer

On the Platform-Independent Layer the models representing the test cases are independent of any particular testing framework or testing technology. On this level of abstraction, we distinguish between two types of models, the Model of the Abstract Tests and the System Behavior Model.

Model of the Abstract Tests: This model is considered to be platform-independent as it is independent of any concrete testing frameworks. Standardized languages like UML Testing Profile (UTP) [36] and Test Description Language (TDL) [18] are used for modeling the abstract tests.

System Behavior Model: On the highest level of abstraction, we foresee the System Behavior Model that represents a compact representation of the expected behavior of the system. Behavioral diagrams like the UML activity or sequence diagram, or state machines can be used to represent the expected behavior of the system.

3.2 Activities

Activities in the test case reengineering horseshoe model produce or consume appropriate artifacts. In our model shown in Fig. 1, we distinguish between five types of activities: Endogenous Model Transformation, Exogenous Model Transformation, Model-to-Text Transformation, Text-to-Model Transformation, and No Transformation (removal/deletion of particular parts). These activities can be distinguished by the reengineering phase they belonging to. We rely on the classification defined by [13], where the reengineering is defined as process constituted of three phases: Reverse Engineering, Restructuring, and Forward Engineering.

Reverse Engineering. According to [13], Reverse Engineering is the process of analyzing a subject system to create representations of the system in another form or on a higher level of abstraction. When applied in the software testing domain, reverse engineering can be seen as a process of analyzing a test cases to create representations of the test cases in another form or on a higher level of abstraction, e.g., by using test models. In general, reverse engineering can be seen as combination of Model Discovery and Model Understanding [12].

The Model Discovery step relies on syntactical analysis and by using parsers in an automatic text-to-model transformation activity creates a model of the test case source code represented as an Abstract Syntax Tree (AST) [9]. The obtained model is known as initial model because it has a direct correspondence to the test cases. Model Discovery actually bridges the technical space of the test cases and the modelware technical space of MDE. The tools used to discover this models are known as model discoverers and for each specific language, an appropriate model discoverer is needed.

Model Understanding is a model-to-model transformation activity, or a chain of such activities, which takes the initial models, applies semantic mappings and generates derived models. This transformation performs a semantic mapping and results in a model of higher level of abstraction. In our reengineering model, we split the Model Understanding in three sub-activities: Firstly, the initial models are explored by navigating through their structure in an activity called Test Case Understanding. Model elements which represent test relevant concepts like test suite, test case or assertion are identified and then, by applying a model-to-model transformation, a test model of executable test cases is obtained. A model on this level is platform-specific which means that it is specific to a certain test technology and it is an instance of a metamodel of a particular testing framework (e.g., jUnit, MSUnit or TTCN-3). By applying the Test Abstraction activity, which is a model-to-model transformation as well, one can obtain a model of the abstract test cases which is platform-independent. As a platform-independent representation, UML Testing Profile (U2TP)  [36] or Test Description Language (TDL) [18] may be used. The System Behavior Recovery activity is applied in order to obtain the highest possible level of abstraction defined by our reengineering model, the System Behavior Model which is a compact representation of the expected behavior of the system. Regarding the technical side of the model-to-model transformations, there are different transformation languages that can be used, e.g., QVT [35], JDT [1] or ATL [27].

Forward Engineering. As defined by [13], Forward Engineering is “the traditional process of moving from high-level abstractions and logical, implementation-independent designs to the physical implementation of a system”. In the field of software testing, this can be paraphrased as a process of moving of high-level test abstractions and logical implementation-independent design to the physical implementation of the test cases. The complete Forward Engineering side can be seen as Model-Based Testing, more specifically Model-Driven Testing [16, 23]. The test models are used as input for a chain of model-to-model transformations, ending with a model-to-text transformation, which provides the test code as output.

Abstract Tests Derivation takes as input the System Behavior Model and produces Model of Abstract Test Cases, a platform-independent model of the test cases. Then, by applying Test Concretization a Model of Executable Test Cases is obtained. For example, in case of UTP as model of the abstract tests and TTCN-3 as a model of executable tests, the mapping presented in [39] can be used as a base in the Test Concretization activity. This model is already specific for a particular testing framework and can be used for the generation of the Test Code by executing the Test Code Generation activity, which is a model-to-text transformation.

The tools which support forward engineering are known as generators and for each specific target platform, an appropriate generator is needed. Custom code generators can be built by using Xtend [8] which is a statically typed programming language which offers features like template expressions and intelligent space management.

Restructuring. According to [13], Restructuring is “the transformation from one representation form to another at the same relative abstraction level, while preserving the subject systems external behavior (functionality and semantics)”. In the testing domain, we define test restructuring as the transformation from one representation to another at the same relative abstraction level, while preserving the “semantics” of the tests. With “semantics” we mean the functionality that is being checked by a particular test. This activity has been foreseen on both the Test Model as well as on the Model of Abstract Test Cases. The Restructuring activity is of course influenced by the target testing environment, testing tool, or by requirements on improving the quality of the test cases (e.g., maintainability). However, it could also be influenced by the changes that happen in the system migration. Since these changes may be relevant for the test models, they have to be reflected on the tests as well.

The Reimplementation is a Text-to-Text transformation which is performed by developers or testers manually, firstly observing the existing test cases an then implementing them for the target testing environment.

The Language Transformation also known as AST-based transformation [29], defines a direct mapping between the Models of the Test Code. This mapping between the original and the migrated model, is actually a mapping between the programming languages and the testing frameworks in which the test cases are implemented.

The Framework Transformation activity is performed on a higher level of abstraction and it defines a mapping directly between two testing frameworks, i.e., it defines a mapping between the testing concepts inside the original an the target testing framework.

The Enrichment activity is applicable to various models, e.g., Models of Executable Tests, Models of Abstract Tests or Test Models. By using annotation, one can insert an additional information to the tests.

The Removal activity is used to specify on which part of the test case code a transformation should not be performed. Due to the evolutionary development of the system as well as of the test cases, inconsistencies between the test code and the system code may exist, i.e., it may happen that no longer supported features or non-existing parts of the system are still being tested. Consequently, on those parts of the test code with obsolete test cases, a transformation is not performed. Intuitively, this activity compared to all other, is the only one that does not produce output.

Seen from the testing perspective, these three abstraction layers can be mapped to two testing abstraction layers. Namely, the Platform-Independent Layer is actually the Test Design Layer, where the initial test models are designed, the System Behavior Model of the SUT, and the Model of the Abstract Tests. Then, comes Test Implementation, where Model of the Executable Tests are derived which represent an implementation with concrete test data.

3.3 Tools

In order to support the previously introduced activities, thus enabling a (semi-) automatic transformation, we foresee in total three types of tools that are needed. Firstly, a parser is needed to obtain the initial models out of the textual artifacts, i.e, to perform the Model Discovery activity. Then, in a series of model-to-model transformations, which are specified by transformation rules, initial models are obtained. There rules are then executed by an Transformation Rule Engine. Finally, a Test Code Generator is needed to perform the model-to-text transformation by executing the test code generation rules previously specified, thus generating the test code for the migrated test cases.

4 Industrial Case Study

Our reengineering horseshoe model was applied in the context of an industrial project with the main goal of migrating parts of the well-known Eclipse Modeling Framework (EMF) [2] along with the Object Constraint Language (OCL) [6] from Java to C#. The old system is thus a Java program representing a subset of EMF and OCL, while the migrated system is an implementation of this subset as a corresponding C# program.

4.1 Migration Context

The primary change implemented by the system migration, besides switching from Java to C#, was to change from a Just-In-Time (JIT) compilation in the old system to an Ahead-Of-Time (AOT) compilation of OCL constraints in the migrated system.

As EMF and OCL are both well-tested frameworks, with all test cases being available on public code repositories [7], a major goal of the case study was to reuse the old OCL test cases to validate the migrated OCL functionality in the target environment.

It can be distinguished between two main types of source artifacts that have to be migrated, the test cases and the interlayer testing framework TestOCL. All an all, 13 different test suite are present, each of them addressing different functional aspects of OCL. On the first look, most of the test cases have similar structure. This characteristic is important, since we aim at automating the migration process of the test cases. The test cases that are following same structure are selected for automated migration. For those test cases that is difficult some repeating structure to identify, manual implementation is a more suitable solution. Additionally, an interlayer testing framework TestOCL exists in order to simplify the test cases. It provides domain-specific assert functions for the OCL domain, e.g., assertQueryTrue.

Regarding the language, the source system is implemented in Java under Java Platform. Regarding the architecture, OCL is implemented in that a native OCL expressions are specified and using the JAVA API they are evaluated. The language in the target environment is C# under the .net platform. The OCL functionalities are implemented directly in the corresponding classes that are specified in the migrated system. Regarding the transformation characteristics, the activities we ranging from manual to completely automatic. Those transformations that were performed automatically, have been formalized using transformation rules specified in Java. The process of transformation was supported by implementing transformation rules based on Xtend [8]. The thing of highest importance for the reflection of the changes to the test cases is the transformation rules specified for the transformation of OCL expression to C#. The change type can be seen as semantic-preserving, since the complete functionality of the OCL should be supported by the migrated system.

The source test environment is JUnit, with an additional interlayer extension, the TestOCL. The test cases are unit level test cases, separated in different test suites according the functionality they are addressing. The anatomy of the test cases in the target environment is that a single test case may contain multiple assertions, normally specified using the assert functions provided by the TestOCL extension. The complete change of the environment regarding language, framework, and architecture, implies a lot of changes in the testing context. The target environment language, in this particular case C#, implies usage of a C# Unit testing framework, MSUnit.

According to this context information, we came up with the migration method shown in Fig. 2. It is an instance of the previously introduced test case reengineering horseshoe (Fig. 1).

4.2 Implementation

To support the test case migration method, two Eclipse plug-ins [2] were developed,  TestCase2TestModel and TestModel2TestCase. The names of the two plug-ins pretty much reveal what exactly the particular plug-in does. The TestCase2TestModel plug-in supports the activities on the left-hand side of the reengineering horseshoe, i.e., supports the reverse engineering activities, thus extracting test models from the test cases. The activities on the right-hand side of the migration horseshoe, i.e., the forward engineering activities, are supported by the TestModel2TestCase plug-in which takes as input the test models and generates test code. Both plug-ins work in a completely automated way.

TestCase2TestModel. This plug-in supports the Reverse Engineering phase from the test case reengineering method. Consequently, it covers both the Model Discovery as well as Test Case Understanding activities. For Model Discovery, it uses the JDT Parser [1] to do a text-to-model transformation. It takes as input jUnit [3] test cases, i.e., a Java code and it produces an abstract syntax tree (AST). The AST is the input for the second activity, where out of the language-specific model elements, testing relevant concepts are identified and extracted. Out of this information, Models of the Executable Test Cases are produced which conform to the xUnit meta-model [33]. Thus, the newly obtained models contain model elements like test suite, test case or assertion, making the information explicit from testing point of view.

Figure 2.
figure 2

The case study migration method: an instance of the test case reengineering horseshoe model

Full size image

TestModel2TestCase. This plug-in covers the activities from Restructuring and Forward Engineering Phase. The transformation of OCL expressions from to C# has already been implemented. As the action in our test cases is actually to evaluate OCL expressions, we reuse the same transformation, to generate the test code in C#. The plug-in itself is a test code generator written using Xtend [8]. As input it takes the Xunit test models which represent the test suites and the test cases which are the outcome of the first plug-in. Using Xtend templates, test case code in MSUnit was generated for each corresponding test case from the Xunit test models. MS Unit Test Framework [4] was selected as a target testing framework environment for the test cases. As we have said, this plug-in reuses the same transformation from the system migration. Additionally, a special requirement regarding the anatomy of the test cases has to be followed, i.e., the so called “3-As” Pattern had to be applied. The “3-As” Pattern defines that each test case on a code level is consisted of three parts: Arrange, Act, and Assert. For each of the three steps, OCL expressions have been translated to an appropriate C#-based form. By embedding the transformation initially created for the system migration, we were able to transform the test cases in the same way.

The overall result was an automated migration of over 92% of 4000 existing jUnit test cases. It is an open question if the migration of the test cases can further be automatized. 8% of the OCL test cases were not migrated automatically, because these are regression tests that have an irregular structure which complicates an automation.

5 Related Work

Regarding related work, we analyzed different research areas, software migration projects, and migration frameworks. Here, research areas like model-driven testing, test case reengineering, test mining and software migration are discussed. The area of model-driven testing includes already some proved methods that address the automation of the test case generation in a quite efficient way. In the work presented in [23], the authors aim to benefit from the separation of PIMs and PSMs in the generation and execution of tests by redefining the classical model-based tasks. Dai [14] proposes a model-driven testing methodology which takes as input UML system models and transforms them to test-specific models as instances of the UML Testing Profile. Javed et al. [26] propose a generic method for model-driven testing that relies on xUnit platform independent model. The work of Sousa et al. [26] represents an improvement of the work of Javed et al. in a way that they use a new test metamodel on the platform-independent level, whereas the xUnit metamodel is considered as a platform-specific. Lamancha et al. [30, 31] propose also a model-driven testing method that relies on UML Testing Profile. Moreover, they present concrete transformations using the QVT transformation language. All of this methods are related to our work as they address the forward engineering side of our reengineering model. Regarding the reverse engineering side, we have identified also some existing work in the area known as test case reengineering. Hungar et al. [24] extract models out of test cases by means of automata learning. In [25], test models are synthesized from test cases by threating all test cases as a linear model and merging the corresponding states. The work of Werner et al. [42] goes in the similar direction and constructs trace graphs out of test cases to represent the system behavior.  [15, 34] go in the similar direction, and by exploiting the knowledge in the existing test cases written in Gherkin format, and extracting models in terms of finite state machines or state-flow graphs. Doing so, they aim to convert existing projects to model-based testing projects.

In a lot of software migration projects the importance of reusing test cases has been detected and an effort has been made to enable test case reuse. In the SOAMIG [32] migration project for example, existing test cases are used for the analysis of the legacy system. MoDisco [12] is a generic and extensible framework devoted to Model-Driven Reverse Engineering. However, migration of test cases is still not addressed by this framework. The Artist [32] project proposes model-based modernization, by employing MDE techniques to automate the reverse engineering and forward engineering phases. From our point of view, it is the most interesting project as they also advocate migration of the test cases in a model-driven way, i.e., in a similar way the system has been migrated, thus reusing, above all, the developed tools. Our work differs in that way, that we propose a reengineering horseshoe model seen from software testing perspective.

6 Conclusion and Future Work

Inspired by the usage of Model-Driven Engineering and Model-Driven Architecture in software migration, in this paper we propose a test case reengineering horseshoe model which should be a fundamental step in enabling model-driven test case migration. It comprises the basic reengineering activities, Reverse Engineering, Restructuring, and Forward Engineering and artifacts on different levels of abstraction specific for software testing.

The proposed test case reengineering horseshoe model is a generic model with a high flexibility, as it can that can be used in different migration context for the migration of test cases. Employing a discipline like situational method engineering, one can come up with different test case transformation methods suitable for different migration contexts [21]. Therefore, we intend to use our reengineering model as a reference model in a method engineering approach, which would enable, according to a specific context, creation of an suitable migration method. Furthermore, our method relies on higher level of abstraction on well-established standards like the UML Testing Profile (UTP), an OMG standard, and on the especially interesting because of the growing popularity, Test Description Language (TDL), an ETSI standard. By developing appropriate tooling, also a high level of automation can be achieved thus reducing time and cost in the overall migration project. Last but not least, our method should enable co-migration of test cases in an migration scenario. Our main idea regarding this is to make the transformation activities of the reengineering model parameterizable, thus enabling reflection of the relevant transformation activities performed on the system.