Skip to main content
SpringerLink
Log in

Verification of Microservices Using Metamorphic Testing

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11944))

  • 1807 Accesses

Abstract

Microservices architecture is drawing more and more attention recently. By dividing the monolithic application into different services, microservices-based applications are more flexible, scalable and portable than traditional applications. However, the unique characteristics of Microservices architecture have also brought significant challenges for software verification. One major challenge is the oracle problem: in the testing of microservices, it is often very difficult to verify the test result given a test input, due to the features of wide distribution, heterogeneity, frequent changes, and numerous runtime behaviors. To tackle such a challenge, in this paper, we investigate how to apply metamorphic testing into the verification of microservices-based applications, which is a simple yet effective approach to oracle problem. Empirical studies are conducted to evaluate the performance of metamorphic testing based on real-world microservice applications, against the baseline random testing technique with a complete oracle. The results show that in the absence of oracles, metamorphic testing can deliver relatively high failure-detection effectiveness. Our work demonstrates that metamorphic testing is both applicable and effective in addressing the oracle problem for the verification of microservices, similar to many other application domains.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arcuri, A.: Restful API automated test case generation. In: IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 9–20. IEEE (2017)

    Google Scholar 

  2. Bai, X., Dong, W., Tsai, W.T., Chen, Y.: WSDL-based automatic test case generation for web services testing. In: IEEE International Workshop on Service-Oriented System Engineering (SOSE 2005), pp. 207–212. IEEE (2005)

    Google Scholar 

  3. Berners-Lee, T., Fielding, R., Masinter, L.: Uniform resource identifier (URI): generic syntax. Technical report (2004)

    Google Scholar 

  4. Calçado, P.: Building products at soundcloud—Part I: dealing with the monolith (2014). https://tinyurl.com/jxy8yl9. Accessed 23 June 2019

  5. de Camargo, A., Salvadori, I., Mello, R.D.S., Siqueira, F.: An architecture to automate performance tests on microservices. In: Proceedings of the 18th International Conference on Information Integration and Web-Based Applications and Services, pp. 422–429. ACM (2016)

    Google Scholar 

  6. Chakrabarti, S.K., Rodriquez, R.: Connectedness testing of restful web-services. In: Proceedings of the India Software Engineering Conference, pp. 143–152. ACM (2010)

    Google Scholar 

  7. Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: a new approach for generating next test cases. Technical report, HKUST-CS98-01, Department of Computer Science, Hong Kong (1998)

    Google Scholar 

  8. Chen, T.Y., et al.: Metamorphic testing: a review of challenges and opportunities. ACM Comput. Surv. (CSUR) 51(1), 4 (2018)

    Article  Google Scholar 

  9. Chen, T., Ho, J.W., Liu, H., Xie, X.: An innovative approach for testing bioinformatics programs using metamorphic testing. BMC Bioinform. 10(1), 24 (2009)

    Article  Google Scholar 

  10. Crockford, D.: The application/json Media Type for JavaScript Object Notation (JSON). Technical report (2006)

    Google Scholar 

  11. Customers and orders management application. https://github.com/eventuate-tram/eventuate-tram-sagas-examples-customers-and-orders. Accessed 23 June 2019

  12. Event-sourcing + CQRS example application. https://tinyurl.com/y3fqzz8y. Accessed 23 June 2019

  13. Eventuate.io microservice provider. https://eventuate.io/. Accessed 23 June 2019

  14. Fertig, T., Braun, P.: Model-driven testing of restful APIs. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1497–1502. ACM (2015)

    Google Scholar 

  15. Fielding, R., et al.: Hypertext Transfer Protocol - HTTP/1.1. Technical report (1999)

    Google Scholar 

  16. Fielding, R.T., Taylor, R.N.: Architectural styles and the design of network-based software architectures, vol. 7. Irvine Doctoral dissertation, University of California (2000)

    Google Scholar 

  17. Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. ACM SIGPLAN Not. 40, 213–223 (2005)

    Article  Google Scholar 

  18. Goldberg, Y.: Scaling gilt: from monolithic ruby application to distributed Scala micro-services architecture (2014)

    Google Scholar 

  19. Heinrich, R., et al.: Performance engineering for microservices: research challenges and directions (2017)

    Google Scholar 

  20. Heorhiadi, V., Rajagopalan, S., Jamjoom, H., Reiter, M.K., Sekar, V.: Gremlin: systematic resilience testing of microservices. In: International Conference on Distributed Computing Systems (ICDCS), pp. 57–66. IEEE (2016)

    Google Scholar 

  21. Hierons, R.M.: Oracles for distributed testing. IEEE Trans. Softw. Eng. 38(3), 629–641 (2011)

    Article  Google Scholar 

  22. Ihde, S., Parikh, K.: From a monolith to microservices + REST: the evolution of LinkedIn’s service architecture (2015)

    Google Scholar 

  23. Kramer, S.: The biggest thing Amazon got right: the platform. Gigaom, 12 October 2011

    Google Scholar 

  24. Lamela Seijas, P., Li, H., Thompson, S.: Towards property-based testing of RESTful web services (2013)

    Google Scholar 

  25. Le, V., Afshari, M., Su, Z.: Compiler validation via equivalence modulo inputs. ACM SIGPLAN Not. 49, 216–226 (2014)

    Article  Google Scholar 

  26. Lewis, J., Fowler, M.: Microservices (2014). martinfowler.com

  27. Liu, H., Kuo, F., Towey, D., Chen, T.Y.: How effectively does metamorphic testing alleviate the oracle problem? IEEE Trans. Softw. Eng. 40(1), 4–22 (2013)

    Article  Google Scholar 

  28. Mauro, T.: Adopting microservices at netflix: lessons for architectural design (2015). Acesso em 8 2016

    Google Scholar 

  29. Namiot, D., Sneps-Sneppe, M.: On micro-services architecture. Int. J. Open Inf. Technol. 2(9), 24–27 (2014)

    Google Scholar 

  30. Noikajana, S., Suwannasart, T.: An improved test case generation method for web service testing from WSDL-S and OCL with pair-wise testing technique. In: International Computer Software and Applications Conference, vol. 1, pp. 115–123. IEEE (2009)

    Google Scholar 

  31. Panda, A., Sagiv, M., Shenker, S.: Verification in the age of microservices. In: Proceedings of the 16th Workshop on Hot Topics in Operating Systems, pp. 30–36. ACM (2017)

    Google Scholar 

  32. Parsai, A., Murgia, A., Demeyer, S.: LittleDarwin: a feature-rich and extensible mutation testing framework for large and complex Java systems. In: Dastani, M., Sirjani, M. (eds.) FSEN 2017. LNCS, vol. 10522, pp. 148–163. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68972-2_10

    Chapter  Google Scholar 

  33. Pinheiro, P.V.P., Endo, A.T., Simao, A.: Model-based testing of restful web services using UML protocol state machines. In: Brazilian Workshop on Systematic and Automated Software Testing, pp. 1–10 (2013)

    Google Scholar 

  34. Restaurant management application. https://github.com/eventuate-examples/eventuate-examples-restaurant-management. Accessed 23 June 2019

  35. Richardson, C.: Microservices: decomposing applications for deployability and scalability. InfoQ 25, 15–16 (2014)

    Google Scholar 

  36. Savchenko, D., Radchenko, G.: Microservices validation: methodology and implementation. In: 1st Ural Workshop on Parallel, Distributed, and Cloud Computing for Young Scientists, pp. 21–28 (2015)

    Google Scholar 

  37. Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)

    Article  Google Scholar 

  38. Segura, S., Parejo, J.A., Troya, J., Ruiz-Cortés, A.: Metamorphic testing of restful web APIs. IEEE Trans. Softw. Eng. 44(11), 1083–1099 (2017)

    Article  Google Scholar 

  39. Sun, C.A., Wang, G., Mu, B., Liu, H., Wang, Z., Chen, T.Y.: Metamorphic testing for web services: framework and a case study. In: 2011 IEEE International Conference on Web Services, pp. 283–290. IEEE (2011)

    Google Scholar 

  40. Transactional service application. https://github.com/cer/event-sourcing-examples. Accessed 23 June 2019

  41. Villamizar, M., et al.: Evaluating the monolithic and the microservice architecture pattern to deploy web applications in the cloud. In: 2015 10th Computing Colombian Conference (10CCC), pp. 583–590. IEEE (2015)

    Google Scholar 

  42. Hu, W., Yu, H., Liu, X., Chen, X.: Study on REST API test model supporting web service integration. In: 2017 IEEE 3rd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS), pp. 133–138. IEEE (2017)

    Google Scholar 

  43. Yu, D., Jin, Y., Zhang, Y., Zheng, X.: A survey on security issues in services communication of microservices-enabled fog applications. Concurr. Comput. Pract. Exp. 31, e4436 (2018)

    Google Scholar 

  44. Zheng, T., et al.: SmartVM: a SLA-aware microservice deployment framework. World Wide Web 22(1), 275–293 (2019). https://doi.org/10.1007/s11280-018-0562-5

    Article  Google Scholar 

  45. Zheng, X., Julien, C., Podorozhny, R., Cassez, F.: BraceAssertion: runtime verification of cyber-physical systems. In: 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems, pp. 298–306. IEEE (2015)

    Google Scholar 

  46. Zheng, X., Julien, C., Podorozhny, R., Cassez, F., Rakotoarivelo, T.: Efficient and scalable runtime monitoring for cyber-physical system. IEEE Syst. J. 12(2), 1667–1678 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing, Macquarie University, Sydney, NSW, 2109, Australia

    Gang Luo, Xi Zheng & Er Zhuang

  2. School of Information Technology, Deakin University, Geelong, VIC, 3220, Australia

    Dinesh Nagumothu, Ranjith Janapareddi & Xiao Liu

  3. Department of Computer Science and Software Engineering, Swinburne University of Technology, Melbourne, Australia

    Huai Liu

  4. School of Computer Science and Technology, Anhui University, Hefei, China

    Rongbin Xu

  5. School of Information and Electronic Engineering, Zhejiang University of Science and Technology, Hangzhou, China

    Er Zhuang

Authors
  1. Gang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xi Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rongbin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dinesh Nagumothu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ranjith Janapareddi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Er Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xi Zheng .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Software Engineering, Swinburne University of Technology, Hawthorn, Melbourne, VIC, Australia

    Sheng Wen

  2. School of Computer Science, The University of Sydney, Camperdown, NSW, Australia

    Albert Zomaya

  3. Department of Computer Science, St. Francis Xavier University, Antigonish, NS, Canada

    Laurence T. Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Luo, G. et al. (2020). Verification of Microservices Using Metamorphic Testing. In: Wen, S., Zomaya, A., Yang, L. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11944. Springer, Cham. https://doi.org/10.1007/978-3-030-38991-8_10

Download citation

Publish with us

Policies and ethics

Search

Navigation