Abstract
To support fast development cycles in data science, microservice architectures are becoming increasingly important. However, while the design and identification of microservices in transaction-oriented applications are already widely studied, software architects lack support for data science workflows. The identification of microservices for data science workflows differs due to high volume and velocity characteristics.
With this work, we aim to present a multi-metric approach for decomposition of microservice-based data science workflows. First, we select different metrics and evaluate their impact on workflow execution under different workload and data conditions. Within the approach, we provide a software architecture that enables microservice architectures to be deployed concurrently in cloud environments considering microservice design patterns such as orchestration of choreography. This architecture can be used to run real-world experiments, aggregate logs and analyze them in an automated way with respect to our chosen metrics. We evaluated our approach using a real-world data science workflow for automated startup assessments.
Our work has both practical, theoretical and economic implications. Practically, it can support software architects and data scientists in architecting microservices. In this context, it also has implications for MLOps, as microservices can be used to train and deploy ML models. Theoretically, our software architecture can be used for other research comparing microservice architectures. Economically, we also achieve business impact by looking at the cost of microservice architectures based on service activation time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://airflow.apache.org/ (Last accessed 03 August 2022).
- 2.
https://jmeter.apache.org/ (Last accessed 03 August 2022).
- 3.
https://aws.amazon.com/cloudformation/ (Last accessed 03 August 2022).
- 4.
https://www.oasis-open.org/committees/tosca/ (Last accessed 03 August 2022).
References
Assunção, W.K.G., Colanzi, T.E., Carvalho, L., Pereira, J.A., Garcia, A., de Lima, M.J., Lucena, C.: A multi-criteria strategy for redesigning legacy features as microservices: an industrial case study. In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 377–387 (2021)
Athanasopoulos, D., Zarras, A.V., Miskos, G., Issarny, V., Vassiliadis, P.: Cohesion-driven decomposition of service interfaces without access to source code. IEEE Trans. Serv. Comput. 8(4), 550–562 (2015)
Bao, L., Wu, C., Bu, X., Ren, N., Shen, M.: Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Trans. Parallel Distrib. Syst. 30(9), 2114–2129 (2019)
Camilli, M., Colarusso, C., Russo, B., Zimeo, E.: Domain metric driven decomposition of data-intensive applications. In; 2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 189–196 (2020)
Capuano, R., Muccini, H.: A systematic literature review on migration to microservices: a quality attributes perspective. In: IEEE 19th International Conference on Software Architecture Companion (ICSA-C), vol. 19, pp. 120–123 (2022)
Cojocaru, M., Uta, A., Oprescu, A.: Attributes assessing the quality of microservices automatically decomposed from monolithic applications. In: 2019 18th International Symposium on Parallel and Distributed Computing (ISPDC), pp. 84–93 (2019)
Daoud, M., El Mezouari, A., Faci, N., Benslimane, D., Maamar, Z., El Fazziki, A.: A multi-model based microservices identification approach. J. Syst. Archit. 118, 102200 (2021)
Fritzsch, J., Bogner, J., Zimmermann, A., Wagner, S.: From monolith to microservices: a classification of refactoring approaches. In: Bruel, J.M., Mazzara, M., Meyer, B. (eds.) Software Engineering Aspects of Continuous Development and New Paradigms of Software Production and Deployment, vol. 11350. Springer International Publishing, Cham (2019)
Gysel, M., Kölbener, L., Giersche, W., Zimmermann, O.: Service cutter: a systematic approach to service decomposition. In: Aiello, M., Johnsen, E.B., Dustdar, S., Georgievski, I. (eds.) ESOCC 2016. LNCS, vol. 9846, pp. 185–200. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44482-6_12
Hasselbring, W.: Microservices for scalability: keynote talk abstract. In: ICPE 2016: Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering, pp. 133–134 (2016)
Jin, W., Liu, T., Cai, Y., Kazman, R., Mo, R., Zheng, Q.: Service candidate identification from monolithic systems based on execution traces. IEEE Trans. Softw. Eng. 47, 987–1007 (2019)
Klock, S., van der Werf, J.M.E.M., Guelen, J.P., Jansen, S.: Workload-based clustering of coherent feature sets in microservice architectures. In: 2017 IEEE International Conference on Software Architecture (ICSA), pp. 11–20 (2017)
Li, S., et al.: A dataflow-driven approach to identifying microservices from monolithic applications. J. Syst. Softw. 157, 1–16 (2019). http://www.sciencedirect.com/science/article/pii/S0164121219301475
Provost, F., Fawcett, T.: Data science and its relationship to big data and data-driven decision making. Big Data 1(1), 51–59 (2013)
Richardson, C.: Microservice Patterns: With Examples in Java. Manning, Shelter Island (2019)
Schröer, C., Kruse, F., Marx Gómez, J.: A qualitative literature review on microservices identification approaches. In: Dustdar, S. (ed.) SummerSOC 2020. CCIS, vol. 1310, pp. 151–168. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64846-6_9
Schröer, C., Wittfoth, S., Marx Gómez, J.: A process model for microservices design and identification. In: 2021 IEEE International Conference on Software Architecture Companion (ICSA-C), pp. 38–45 (2021)
Shimoda, A., Sunada, T.: Priority order determination method for extracting services stepwise from monolithic system. In: 2018 7th International Congress on Advanced Applied Informatics (IIAI-AAI), pp. 805–810 (2018)
Tamburri, D.A.: Sustainable MLOps: trends and challenges. In: 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), vol. 22, pp. 17–23 (2020)
Zhang, Y., Zhu, L., Xu, X., Chen, S., Tran, A.B.: Data service API design for data analytics. In: Ferreira, J.E., Spanoudakis, G., Ma, Y., Zhang, L.-J. (eds.) SCC 2018. LNCS, vol. 10969, pp. 87–102. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94376-3_6
Acknowledgements
The results, opinions and conclusions expressed in this thesis are not necessarily those of Volkswagen Aktiengesellschaft.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schröer, C., Wittfoth, S., Gómez, J.M. (2023). Multi-metric Approach for Decomposition of Microservice-Based Data Science Workflows. In: Batista, T., Bureš, T., Raibulet, C., Muccini, H. (eds) Software Architecture. ECSA 2022 Tracks and Workshops. ECSA 2022. Lecture Notes in Computer Science, vol 13928. Springer, Cham. https://doi.org/10.1007/978-3-031-36889-9_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-36889-9_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36888-2
Online ISBN: 978-3-031-36889-9
eBook Packages: Computer ScienceComputer Science (R0)