Regression Testing of Web Service: A Systematic Mapping Study

Web service is a widely used implementation technique under the paradigm of Service-Oriented Architecture (SOA). A service-based system is subjected to continuous evolution and regression testing is required to check whether new faults have been introduced. Based on the current scientific work of web service regression testing, this survey aims to identify gaps in current research and suggests some promising areas for further study. To this end, we performed a broad automatic search on publications in the selected electronic databases published from 2000 to 2013. Through our careful review and manual screening, a total of 30 papers have been selected as primary studies for answering our research questions. We presented a qualitative analysis of the findings, including stakeholders, challenges, standards, techniques, and validations employed in these primary studies. Our main results include the following: (1) Service integrator is the key stakeholder that largely impacts how regression testing is performed. (2) Challenges of cost and autonomy issues have been studied heavily. However, more emphasis should be put on the other challenges, such as test timing, dynamics, privacy, quota constraints, and concurrency issues. (3) Orchestration-based services have been largely studied, while little attention has been paid to either choreography-based services or semantic-based services. (4) An appreciable amount of web service regression testing techniques have been proposed, including 48 test case prioritization techniques, 10 test selection techniques, two test suite minimization techniques, and another collaborative technique. (5) Many regression test techniques have not been theoretically proven or experimentally analyzed, which limits their application in large-scale systems. We believe that our survey has identified gaps in current research work and reveals new insights for the future work.