Skip to main content
SpringerLink
Log in

Service-Oriented Architectures Testing: A Survey

  • Chapter
Software Engineering (ISSSE 2007, ISSSE 2008, ISSSE 2006)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5413))

Abstract

Testing of Service Oriented Architectures (SOA) plays a critical role in ensuring a successful deployment in any enterprise. SOA testing must span several levels, from individual services to inter-enterprise federations of systems, and must cover functional and non-functional aspects.

SOA unique combination of features, such as run-time discovery of services, ultra-late binding, QoS aware composition, and SLA automated negotiation, challenge many existing testing techniques. As an example, run-time discovery and ultra-late binding entail that the actual configuration of a system is known only during the execution, and this makes many existing integration testing techniques inadequate. Similarly, QoS aware composition and SLA automated negotiation means that a service may deliver with different performances in different contexts, thus making most existing performance testing techniques to fail.

Whilst SOA testing is a recent area of investigation, the literature presents a number of approaches and techniques that either extend traditional testing or develop novel ideas with the aim of addressing the specific problems of testing service-centric systems. This chapter reports a survey of recent research achievements related to SOA testing. Challenges are analyzed from the viewpoints of different stakeholders and solutions are presented for different levels of testing, including unit, integration, and regression testing. The chapter covers both functional and non-functional testing, and explores ways to improve the testability of SOA.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Paolucci, M., Kawamura, T., Payne, T.R., Sycara, K.: Semantic matching of web services capabilities. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, pp. 333–347. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  2. Bromberg, Y.D., Issarny, V.: INDISS: Interoperable discovery system for networked services. In: Alonso, G. (ed.) Middleware 2005. LNCS, vol. 3790, pp. 164–183. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  3. Pistore, M., Traverso, P.: Assumption-based composition and monitoring of web services. In: Test and Analysis of web Services, pp. 307–335. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  4. Di Nitto, E., Di Penta, M., Gambi, A., Ripa, G., Villani, M.L.: Negotiation of service level agreements: An architecture and a search-based approach. In: Krämer, B.J., Lin, K.-J., Narasimhan, P. (eds.) ICSOC 2007. LNCS, vol. 4749, pp. 295–306. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  5. Baresi, L., Guinea, S.: Towards dynamic monitoring of WS-BPEL processes. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 269–282. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  6. Kephart, J., Chess, D.: The vision of autonomic computing. IEEE Computer (2003)

    Google Scholar 

  7. Hinchey, M.G., Sterritt, R.: Self-managing software. Computer 39, 107–109 (2006)

    Article  Google Scholar 

  8. Turner, M., Budgen, D., Brereton, P.: Turning software into a service. IEEE Computer 36, 38–44 (2003)

    Article  Google Scholar 

  9. Walkerdine, J., Melville, L., Sommerville, I.: Dependability properties of P2P architectures. In: 2nd International Conference on Peer-to-Peer Computing (P2P 2002), Linköping, Sweden, 5-7 September 2002, pp. 173–174 (2002)

    Google Scholar 

  10. Baresi, L., Ghezzi, C., Guinea, S.: Smart Monitors for Composed Services. In: Proc. 2nd International Conference on Service Oriented Computing (ICSOC 2004), pp. 193–202. ACM, New York (2004)

    Chapter  Google Scholar 

  11. Canfora, G., Di Penta, M.: Testing services and service-centric systems: Challenges and opportunities. IT Professional 8, 10–17 (2006)

    Article  Google Scholar 

  12. Milanova, A., Rountev, A., Ryder, B.G.: Parameterized object sensitivity for points-to analysis for Java. ACM Trans. Softw. Eng. Methodol. 14, 1–41 (2005)

    Article  Google Scholar 

  13. Bruno, M., Canfora, G., Di Penta, M., Esposito, G., Mazza, V.: Using test cases as contract to ensure service compliance across releases. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 87–100. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  14. Di Penta, M., Bruno, M., Esposito, G., Mazza, V., Canfora, G.: Web services regression testing. In: Baresi, L., Nitto, E.D. (eds.) Test and Analysis of web Services, pp. 205–234. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  15. Ernst, M.D., Cockrell, J., Griswold, W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. IEEE Trans. Software Eng. 27, 99–123 (2001)

    Article  Google Scholar 

  16. Lorenzoli, D., Mariani, L., Pezzè, M.: Automatic generation of software behavioral models. In: 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, May 10-18, 2008, pp. 501–510 (2008)

    Google Scholar 

  17. Ghezzi, C., Mocci, A., Monga, M.: Efficient recovery of algebraic specifications for stateful components. In: IWPSE 2007: Ninth international workshop on Principles of software evolution, pp. 98–105. ACM, New York (2007)

    Google Scholar 

  18. McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verif. Reliab. 14, 105–156 (2004)

    Article  Google Scholar 

  19. Wegener, J., Baresel, A., Sthamer, H.: Evolutionary test environment for automatic structural testing. Information & Software Technology 43, 841–854 (2001)

    Article  Google Scholar 

  20. Di Penta, M., Canfora, G., Esposito, G., Mazza, V., Bruno, M.: Search-based testing of service level agreements. In: Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2007, London, England, UK, July 7-11, 2007, pp. 1090–1097 (2007)

    Google Scholar 

  21. Bertolino, A., Gao, J., Marchetti, E., Polini, A.: Systematic generation of XML instances to test complex software applications. In: Guelfi, N., Buchs, D. (eds.) RISE 2006. LNCS, vol. 4401, pp. 114–129. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  22. Bertolino, A., Gao, J., Marchetti, E., Polini, A.: TAXI - a tool for XML-based testing. In: 29th International Conference on Software Engineering (ICSE 2007), Minneapolis, MN, USA, May 20-26, 2007, pp. 53–54 (2007)

    Google Scholar 

  23. Ostrand, T., Balcer, M.: The category-partition method for specifying and generating functional tests. Communications of the Association for Computing Machinery 31 (1988)

    Google Scholar 

  24. Bai, X., Dong, W., Tsai, W.T., Chen, Y.: Wsdl-based automatic test case generation for web services testing. In: IEEE International Workshop on Service-Oriented System Engineering (SOSE), pp. 215–220. IEEE Computer Society, Los Alamitos (2005)

    Google Scholar 

  25. Conroy, K., Grechanik, M., Hellige, M., Liongosari, E., Xie, Q.: Automatic test generation from GUI applications for testing Web services. In: IEEE International Conference on Software Maintenance, ICSM 2007, pp. 345–354 (2007)

    Google Scholar 

  26. Li, Z., Sun, W., Jiang, Z.B., Zhang, X.: BPEL4WS unit testing: Framework and implementation. In: 2005 IEEE International Conference on web Services (ICWS 2005), Orlando, FL, USA, 11-15 July 2005, pp. 103–110 (2005)

    Google Scholar 

  27. Yuan, Y., Li, Z., Sun, W.: A graph-search based approach to BPEL4WS test generation. In: Proceedings of the International Conference on Software Engineering Advances (ICSEA 2006), Papeete, Tahiti, French Polynesia, October 28 - November 2, 2006, p. 14 (2006)

    Google Scholar 

  28. Tsai, W.T., Chen, Y., Paul, R.A., Liao, N., Huang, H.: Cooperative and group testing in verification of dynamic composite web services. In: 28th International Computer Software and Applications Conference (COMPSAC 2004), Design and Assessment of Trustworthy Software-Based Systems, Hong Kong, China, Proceedings, 27-30 September 2004, pp. 170–173 (2004)

    Google Scholar 

  29. Bucchiarone, A., Melgratti, H., Severoni, F.: Testing service composition. In: Proceedings of the 8th Argentine Symposium on Software Engineering (ASSE 2007) (2007)

    Google Scholar 

  30. Canfora, G., Di Penta, M., Esposito, R., Villani, M.L.: A framework for QoS-aware binding and re-binding of composite Web services. Journal of Systems and Software (in press, 2008)

    Google Scholar 

  31. Canfora, G., Di Penta, M., Esposito, R., Villani, M.L.: An approach for QoS-aware service composition based on genetic algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2005, Proceedings, Washington DC, USA, June 25-29, 2005, pp. 1069–1075. ACM Press, New York (2005)

    Chapter  Google Scholar 

  32. Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Publishing Company, Reading (2000)

    Google Scholar 

  33. Tsai, W.T., Paul, R.J., Song, W., Cao, Z.: Coyote: An XML-based framework for Web services testing. In: 7th IEEE International Symposium on High-Assurance Systems Engineering (HASE 2002), Tokyo, Japan, 23-25 October 2002, pp. 173–176 (2002)

    Google Scholar 

  34. Bertolino, A., Polini, A.: The audition framework for testing Web services interoperability. In: 31st EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO-SEAA 2005), Porto, Portugal, 30 August - 3 September 2005, pp. 134–142. IEEE Computer Society, Los Alamitos (2005)

    Chapter  Google Scholar 

  35. Mei, L., Chan, W.K., Tse, T.H.: Data flow testing of service-oriented workflow applications. In: 30th International Conference on Software Engineering (ICSE 2008), Leipzig, Germany, May 10-18, 2008, pp. 371–380 (2008)

    Google Scholar 

  36. Bartolini, C., Bertolino, A., Marchetti, E., Parissis, I.: Data flow-based validation of web services compositions: Perspectives and examples. In: de Lemos, V.R., Di Giandomenico, F., Muccini, H., Gacek, C., Vieira, M. (eds.) Architecting Dependable Systems. Springer, Heidelberg (2008)

    Google Scholar 

  37. Walkerdine, J., Hutchinson, J., Sawyer, P., Dobson, G., Onditi, V.: A faceted approach to service specification. In: International Conference on Internet and web Applications and Services (ICIW 2007), Le Morne, Mauritius, May 13-19, 2007, p. 20 (2007)

    Google Scholar 

  38. Colombo, M., Di Nitto, E., Di Penta, M., Distante, D., Zuccalà, M.: Speaking a common language: A conceptual model for describing service-oriented systems. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 48–60. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  39. Ricca, F., Torchiano, M., Di Penta, M., Mariano Ceccato, P.T.: Using acceptance tests as a support for clarifying requirements: A series of experiments. Information and Software Technology (in press, 2008)

    Google Scholar 

  40. Mugridge, R., Cunningham, W.: Fit for Developing Software: Framework for Integrated Tests. Prentice-Hall, Englewood Cliffs (2005)

    Google Scholar 

  41. Dai, G., Bai, X., Wang, Y., Dai, F.: Contract-based testing for web services. In: 31st Annual International Computer Software and Applications Conference (COMPSAC 2007), Beijing, China, 24-27 July 2007, pp. 517–526 (2007)

    Google Scholar 

  42. Meyer, B.: Object-Oriented Software Construction, 2nd edn. Prentice-Hall, Englewood Cliffs (1997)

    MATH  Google Scholar 

  43. Ruth, M., Tu, S.: Towards automating regression test selection for Web services. In: Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007, pp. 1265–1266 (2007)

    Google Scholar 

  44. Ruth, M., Oh, S., Loup, A., Horton, B., Gallet, O., Mata, M., Tu, S.: Towards automatic regression test selection for Web services. In: 31st Annual International Computer Software and Applications Conference (COMPSAC 2007), Beijing, China, 24-27 July 2007, pp. 729–736 (2007)

    Google Scholar 

  45. Rothermel, G., Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 173–210 (1997)

    Article  Google Scholar 

  46. Martin, E., Basu, S., Xie, T.: WebSob: A tool for robustness testing of web services. In: 29th International Conference on Software Engineering (ICSE 2007), Minneapolis, MN, USA, May 20-26, 2007, pp. 65–66 (2007)

    Google Scholar 

  47. Martin, E., Basu, S., Xie, T.: Automated testing and response analysis ofweb services. In: 2007 IEEE International Conference on web Services (ICWS 2007), Salt Lake City, Utah, USA, July 9-13, 2007, pp. 647–654 (2007)

    Google Scholar 

  48. Csallner, C., Smaragdakis, Y.: JCrasher: an automatic robustness tester for Java. Softw. Pract. Exper. 34, 1025–1050 (2004)

    Article  Google Scholar 

  49. Fu, C., Ryder, B.G., Milanova, A., Wonnacott, D.: Testing of Java Web services for robustness. In: Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2004, Boston, Massachusetts, USA, July 11-14, 2004, pp. 23–34 (2004)

    Google Scholar 

  50. Canfora, G., Di Penta, M., Esposito, R., Perfetto, F., Villani, M.L.: Service composition (re)Binding driven by application–specific qoS. In: Dan, A., Lamersdorf, W. (eds.) ICSOC 2006. LNCS, vol. 4294, pp. 141–152. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  51. Voas, J.M.: Fault injection for the masses. IEEE Computer 30, 129–130 (1997)

    Article  Google Scholar 

  52. Offutt, J., Xu, W.: Generating test cases for Web services using data perturbation. In: SIGSOFT Softw. Eng. Notes - SECTION: Workshop on testing, analysis and verification of Web services (TAV-WEB), vol. 29, pp. 1–10 (2004)

    Google Scholar 

  53. Beizer, B.: Software Testing Techniques, 2nd edn. International Thomson Computer Press (1990)

    Google Scholar 

  54. Looker, N., Munro, M., Xu, J.: Ws-fit: A tool for dependability analysis of Web services. In: Proceedings of 28th International Computer Software and Applications Conference (COMPSAC 2004), Design and Assessment of Trustworthy Software-Based Systems, Hong Kong, China, 27-30 September 2004, pp. 120–123 (2004)

    Google Scholar 

  55. Tsai, W.T., Gao, J., Wei, X., Chen, Y.: Testability of software in service-oriented architecture. In: 30th Annual International Computer Software and Applications Conference (COMPSAC 2006), Chicago, Illinois, USA, 17-21 September 2006, pp. 163–170 (2006)

    Google Scholar 

  56. Tsai, W.T., Paul, R.J., Wang, Y., Fan, C., Wang, D.: Extending WSDL to facilitate Web services testing. In: 7th IEEE International Symposium on High-Assurance Systems Engineering (HASE 2002), Tokyo, Japan, 23-25 October 2002, pp. 171–172 (2002)

    Google Scholar 

  57. Tsai, W.T., Paul, R.J., Cao, Z., Yu, L., Saimi, A.: Verification of Web services using an enhanced UDDI server. In: Proceedings of the Eighth International Workshop on Object-Oriented Real-Time Dependable Systems, pp. 131–138 (2003)

    Google Scholar 

  58. Heckel, R., Mariani, L.: Automatic conformance testing of web services. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 34–48. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  59. Heckel, R., Lohmann, M.: Towards contract-based testing of web services. Electr. Notes Theor. Comput. Sci. 116, 145–156 (2005)

    Article  Google Scholar 

  60. Bai, X., Xu, D., Dai, G.: Dynamic reconfigurable testing of service-oriented architecture. In: 31st Annual International Computer Software and Applications Conference (COMPSAC 2007), Beijing, China, 24-27 July 2007, pp. 368–378 (2007)

    Google Scholar 

  61. Bertolino, A., De Angelis, G., Polini, A.: A QoS test-bed generator for web services. In: Baresi, L., Fraternali, P., Houben, G.-J. (eds.) ICWE 2007. LNCS, vol. 4607, pp. 17–31. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  62. Bertolino, A., De Angelis, G., Frantzen, L., Polini, A.: Model-Based Generation of Testbeds for Web Services. In: Suzuki, K., Higashino, T., Ulrich, A., Hasegawa, T. (eds.) TestCom/FATES 2008. LNCS, vol. 5047, pp. 266–282. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  63. Canfora, G., Di Penta, M.: SOA: Testing and self-checking. In: Keynote speech at the International Workshop on Web Services - Modeling and Testing (WS-MATE 2006) (2006)

    Google Scholar 

  64. Forestiero, A., Mastroianni, C., Papadakis, H., Fragopoulou, P., Troisi, A., Zimeo, E.: A scalable architecture for discovery and composition in P2P service networks. In: Grid Computing: Achievements and Prospects. Springer, Heidelberg (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RCOST - Research Centre on Software Technology, University of Sannio, Palazzo ex Poste, Via Traiano, 82100, Benevento, Italy

    Gerardo Canfora & Massimiliano Di Penta

Authors
  1. Gerardo Canfora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Massimiliano Di Penta
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Matematica e Informatica, Università di Salerno, Via Ponte Don Melillo, 84084, Fisciano, SA, Italy

    Andrea De Lucia

  2. Dipartimento di Matematica e Informatica (DMI), Università di Salerno, Via Ponte Don Melillo, 84084, Fisciano, SA, Italy

    Filomena Ferrucci

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Canfora, G., Di Penta, M. (2009). Service-Oriented Architectures Testing: A Survey. In: De Lucia, A., Ferrucci, F. (eds) Software Engineering. ISSSE ISSSE ISSSE 2007 2008 2006. Lecture Notes in Computer Science, vol 5413. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-95888-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-95888-8_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-95887-1

  • Online ISBN: 978-3-540-95888-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation