Skip to main content

Advertisement

SpringerLink
Log in

A performance modeling framework for microservices-based cloud infrastructures

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Microservice architectures (MSAs) can increase the performance of distributed systems and enable better resource allocation by sharing underlying resources among multiple microservices (MSs). One of the main advantages of MSAs is the ability to leverage the elasticity provided by an infrastructure so that only the most demanding services are scaled, which can contribute to efficient allocation of processing resources. A major problem in allocating resources to microservices is determining a set of auto-scaling parameters that will result in all microservices meeting specific service level agreements (SLAs). Since the space of feasible configurations can be vast, manually determining a combination of parameter values that will result in all SLAs being met is complex and time consuming. In addition, the performance overhead caused by running microservices concurrently and the overhead caused by the VM instantiation process must also be evaluated. Another problem is that microservices can suffer performance degradation due to resource contention, which depends on how microservices are distributed across servers. To address the aforementioned issues, this paper proposes the modeling of these infrastructures and their auto-scaling mechanisms in a private cloud using stochastic Petri nets (SPNs), the non-dominated sorting genetic algorithm II (NSGA-II), one of the most popular evolutionary algorithms for multiobjective optimization (MOO), and random forest regression (RFR), an ensemble-learning-based method, to identify critical trade-offs between performance and resource consumption considering all deployed MSs. The SPN-based model is capable of representing both instantiation of elastic VMs and a pool of instantiated elastic VMs where only containers are started. The analytical framework enables service providers (SPs) to estimate performance metrics considering configurations that satisfy all performance constraints, use of elastic VMs, discard rate, discard probability, throughput, response time, and corresponding cumulative distribution functions (CDFs). These metrics are critical because they make it possible to estimate the time required to process each request, the number of requests processed in a time interval, the number of requests rejected, and the utilization of resources. The framework was validated with 95% confidence interval (CI) using a real-world testbed. Two case studies were used to investigate its feasibility by evaluating its application in a real scenario. We noticed a significant improvement in performance when using a pool of elastic VMs, where throughput improved by 21.5% and the number of discarded requests decreased by 70%. The application of the framework can help in finding optimized solutions that support both infrastructure planning and online performance prediction, and enable trade-off analyses considering different scenarios and constraints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availibility

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Notes

  1. https://thrift.apache.org/.

  2. https://www.openstack.org/.

  3. https://cloudstack.apache.org/.

  4. https://www.modcs.org/.

  5. https://cloudsimplus.org/.

  6. https://jmeter.apache.org/.

  7. http://opencv.org/.

References

  1. Fowler and Lewis (2014) “Microservices,” https://martinfowler.com/articles/microservices.html

  2. Indrasiri K (2018) Microservices for the enterprise : designing, developing, and deploying. Apress, New York

    Book  Google Scholar 

  3. Newman S (2015) Building microservices: designing fine-grained systems. O’Reilly Media, Sebastopol, CA

    Google Scholar 

  4. Villamizar M, Garcés O, Castro H, Verano M, Salamanca L, Casallas R, Gil S (2015) Evaluating the monolithic and the microservice architecture pattern to deploy web applications in the cloud, In: 2015 10th Computing Colombian Conference (10CCC), pp 583–590

  5. Linthicum DS (2017) Connecting fog and cloud computing. IEEE Cloud Comput 4(2):18–20

    Article  Google Scholar 

  6. Barve Y, Shekhar S, Chhokra A, Khare S, Bhattacharjee A, Sun H, Gokhale A, Kang z (2019) Fecbench: A holistic interference-aware approach for application performance modeling, 06

  7. Dhillon JS, Purini S, Kashyap S (2013) Virtual machine coscheduling: A game theoretic approach, In: 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing, pp 227–234

  8. Ismail BI, Jagadisan D, Khalid MF (2011) Determining overhead, variance & isolation metrics in virtualization for iaas cloud, In: Lin SC, Yen E, (Eds) Data driven e-science, Springer, New York, pp 315–330

  9. da Silva Pinheiro TF, Silva FA, Fé I, Kosta S, Maciel P (2018) Performance prediction for supporting mobile applications’ offloading. J Supercomput 74(8):4060–4103. https://doi.org/10.1007/s11227-018-2414-6

    Article  Google Scholar 

  10. Pereira P, Araujo J, Torquato M, Dantas J, Melo C, Maciel P (2020) Stochastic performance model for web server capacity planning in fog computing. J Supercomput 76(12):9533–9557. https://doi.org/10.1007/s11227-020-03218-w ([Online])

    Article  Google Scholar 

  11. Pereira P, Araujo J, Melo C, Santos V, Maciel P (2021) Analytical models for availability evaluation of edge and fog computing nodes. J Supercomput 77(9):9905–9933. https://doi.org/10.1007/s11227-021-03672-0 ([Online])

    Article  Google Scholar 

  12. Pereira P, Melo C, Araujo J, Dantas J, Santos V, Maciel P (2021) Availability model for edge-fog-cloud continuum: an evaluation of an end-to-end infrastructure of intelligent traffic management service. J Supercomput 78(3):4421–4448. https://doi.org/10.1007/s11227-021-04033-7 ([Online])

    Article  Google Scholar 

  13. Clemente D, Pereira P, Dantas J, Maciel P (2022) Availability evaluation of system service hosted in private cloud computing through hierarchical modeling process. J Supercomput 78(7):9985–10024. https://doi.org/10.1007/s11227-021-04217-1 ([Online])

    Article  Google Scholar 

  14. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

  15. Mitchell M (1998) An introduction to genetic algorithms. Complex Adaptive Systems. Bradford Books, Cambridge, MA

    Book  MATH  Google Scholar 

  16. Mills K, Filliben J, Dabrowski C (2011) Comparing vm-placement algorithms for on-demand clouds, In: IEEE Third International Conference on Cloud Computing Technology and Science, pp 91–98

  17. Hu Y, Laat C, Zhao Z (2019) Optimizing service placement for microservice architecture in clouds. Appl Sci 9:4663

    Article  Google Scholar 

  18. Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/a:1010933404324 ([Online])

    Article  MATH  Google Scholar 

  19. Bao L, Wu C, Bu X, Ren N, Shen M (2019) Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Trans Parallel Distrib Syst 30(9):2114–2129

    Article  Google Scholar 

  20. Khazaei H, Barna C, Beigi-Mohammadi N, Litoiu M (2016) Efficiency analysis of provisioning microservices. In: IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pp 261–268

  21. Gribaudo M, Iacono M, Manini D (2018) Performance evaluation of replication policies in microservice based architectures, In: Proceedings of the Ninth International Workshop on the Practical Application of Stochastic Modelling (PASM), Electronic Notes in Theoretical Computer Science, vol. 337, pp. 45–65. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1571066118300379

  22. Li Q, Li B, Mercati P, Illikkal R, Tai C, Kishinevsky M, Kozyrakis C (2021) Rambo: Resource allocation for microservices using Bayesian optimization. IEEE Comput Archit Lett 20(1):46–49

    Article  Google Scholar 

  23. Merkouche S, Bouanaka C (2022) A proactive formal approach for microservice-based applications auto-scaling, In: Belala F, Benchikha F, Boufaïda Z, Smaali S (eds) Proceedings of The 11th Seminary of Computer Science Research at Feminine (RIF 2022) LIRE laboratory, constantine 2 University- Abdelhamid Mehri, Constantine, Algeria, March 10, 2022, ser. CEUR Workshop Proceedings, vol. 3176. CEUR-WS.org, pp. 15–28. [Online]. Available: http://ceur-ws.org/Vol-3176/paper2.pdf

  24. El Kafhali S, El Mir I, Salah K, Hanini M (2020) Dynamic scalability model for containerized cloud services. Arabian J Sci Eng 45:10693–10708

    Article  Google Scholar 

  25. Molloy MK (1982) Performance analysis using stochastic petri nets. IEEE Trans Comput 31:913–917

    Article  Google Scholar 

  26. Maciel PRM (2022) Performance, reliability, and availability evaluation of computational systems, volume I: performance and background. Taylor & Francis, Chapman and Hall/CRC. [Online]. Available: https://www.routledge.com/Performance-Reliability-and-Availability-Evaluation-of-Computational/Maciel/p/book/9781032295374

  27. Salah T, Zemerly MJ, Yeun CY, Al-Qutayri M, Al-Hammadi Y (2017) Performance comparison between container-based and vm-based services, In: 2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), pp 185–190

  28. Ueda T, Nakaike T, Ohara M (2016) Workload characterization for microservices. In: International Symposium on Workload Characterization (IISWC), pp 1–10, IEEE

  29. Joy AM (2015) Performance comparison between linux containers and virtual machines. In: International Conference on Advances in Computer Engineering and Applications, pp. 342–346

  30. Felter W, Ferreira A, Rajamony R, Rubio J (2015) An updated performance comparison of virtual machines and linux containers. In: IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) pp 171–172

  31. Singh V, Peddoju SK (2017) Container-based microservice architecture for cloud applications, In: 2017 International Conference on Computing, Communication and Automation (ICCCA), pp 847–852

  32. Villamizar M, Garcés O, Ochoa L, Castro H, Salamanca L, Verano M, Casallas R, Gil S, Valencia C, Zambrano A, Lang M, (2016) Infrastructure cost comparison of running web applications in the cloud using aws lambda and monolithic and microservice architectures, In: 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp 179–182

  33. Lin M, Xi J, Bai W, Wu J (2019) Ant colony algorithm for multi-objective optimization of container-based microservice scheduling in cloud, IEEE Access, vol. 7, pp 83088–83100

  34. Jindal A, Podolskiy V, Gerndt M (2019) Performance modeling for cloud microservice applications. In: Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering, ser. ICPE ’19. Association for Computing Machinery, New York, NY, USA, pp 25-32. [Online]. Available: https://doi.org/10.1145/3297663.3310309

  35. Auer F, Lenarduzzi V, Felderer M, Taibi D (2021) From monolithic systems to microservices: An assessment framework, Inf Softw Technol, 137:106600. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950584921000793

  36. Bauer A, Lesch V, Versluis L, Ilyushkin A, Herbst N, Kounev S (2019) Chamulteon: coordinated auto-scaling of micro-services, 07

  37. Khazaei H, Ravichandiran R, Park B, Bannazadeh H, Tizghadam A, Leon-Garcia A (2017) Elascale: autoscaling and monitoring as a service, In: Proceedings of the 27th Annual International Conference on Computer Science and Software Engineering, ser. CASCON ’17. USA: IBM Corporation, pp 234–240

  38. Nitto ED, Florio L, Tamburri DA (2020) Autonomic decentralized microservices: the Gru approach and its evaluation. Springer International Publishing, Cham, pp 209–248. https://doi.org/10.1007/978-3-030-31646-4_9 ([Online])

    Book  Google Scholar 

  39. Mao Y, Oak J, Pompili A, Beer D, Han T, Hu P (2017) Draps: Dynamic and resource-aware placement scheme for docker containers in a heterogeneous cluster, In: 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC), pp 1–8

  40. Rossi F, Cardellini V, Presti FL (2020) Hierarchical scaling of microservices in kubernetes. In: IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), pp 28–37

  41. Horovitz S, Arian Y (2018) Efficient cloud auto-scaling with sla objective using q-learning. In: 2018 IEEE 6th International Conference on Future Internet of Things and Cloud (FiCloud), pp 85–92

  42. Rossi F, Nardelli M, Cardellini V (2019) Horizontal and vertical scaling of container-based applications using reinforcement learning. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp 329–338

  43. Saboor A, Hassan MF, Akbar R, Shah SNM, Hassan F, Magsi SA, Siddiqui MA (2022) Containerized microservices orchestration and provisioning in cloud computing: A conceptual framework and future perspectives, Appl Sci, 12(12). [Online]. Available: https://www.mdpi.com/2076-3417/12/12/5793

  44. Zhang R, Zhong A-M, Dong B, Tian F, Li R (2018) Container-vm-pm architecture: A novel architecture for docker container placement. In: Luo M, Zhang L-J (eds) Cloud computing-CLOUD 2018. Springer International Publishing, Cham, pp 128–140

    Chapter  Google Scholar 

  45. Hussein MK, Mousa MH, Alqarni MA (2019) A placement architecture for a container as a service (caas) in a cloud environment, J Cloud Comput, 8(1). [Online]. Available: https://doi.org/10.1186/s13677-019-0131-1

  46. Khan AA, Zakarya M, Khan R, Rahman IU, Khan M, ur Rehman Khan A (2020) An energy, performance efficient resource consolidation scheme for heterogeneous cloud datacenters, J Netw Comput Appl, 150:102497. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1084804519303571

  47. Murata T (1989) Petri nets: properties, analysis and applications. Proc IEEE 77(4):541–580

    Article  Google Scholar 

  48. Marsan MA, Conte G, Balbo G (1987) A class of generalized stochastic petri nets for the performance evaluation of multiprocessor systems. ACM Trans Comput Syst 2(2):93–122. https://doi.org/10.1145/190.191 ([Online])

    Article  Google Scholar 

  49. German R (2000) Performance analysis of communication systems: modeling with non-Markovian stochastic petri nets. John Wiley & Sons, New York

    MATH  Google Scholar 

  50. Trivedi KS (2001) Probability and statistics with reliability, queuing, and computer science applications. John Wiley and Sons, New York

    MATH  Google Scholar 

  51. Maciel PRM (2022) Performance, reliability, and availability evaluation of computational systems, Volume 2: Reliability, availability modeling, measuring, and data analysis. Taylor & Francis, Chapman and Hall/CRC. [Online]. Available: https://www.routledge.com/Performance-Reliability-and-Availability-Evaluation-of-Computational/Maciel/p/book/9781032306407

  52. Mitchell TM (1997) Machine learning. McGraw-Hill, New York

    MATH  Google Scholar 

  53. Bernstein D (2014) Containers and cloud: From lxc to docker to kubernetes. IEEE Cloud Comput 1(3):81–84

    Article  Google Scholar 

  54. Merkel D (2014) Docker: Lightweight linux containers for consistent development and deployment, Linux J, 2014(239)

  55. Peng J, Zhang X, Lei Z, Zhang B, Zhang W, Li Q (2009) Comparison of several cloud computing platforms, In: 2009 Second International Symposium on Information Science and Engineering. IEEE, pp 23–27

  56. Pinheiro TFS, Oliveira D, Matos R, Silva B, Pereira P, Melo C, Oliveira F, Tavares E, Dantas J, Maciel P (2021) The mercury environment: a modeling tool for performance and dependability evaluation, 06

  57. Zakarya M, Gillam L (2017) An energy aware cost recovery approach for virtual machine migration. In: Bañares JÁ, Tserpes K, Altmann J (eds) Economics of grids, clouds, systems, and services. Springer International Publishing, Cham, pp 175–190

    Chapter  Google Scholar 

  58. Lopez-Pires F, Baran B (2015) Virtual machine placement literature review,. [Online]. Available: arXiv:1506.01509

  59. Valderas P, Torres V, Pelechano V (2020) A microservice composition approach based on the choreography of bpmn fragments, Inf Softw Technol, 127:106370. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950584920301397

  60. Jain R (1990) The art of computer systems performance analysis: techniques for experimental design, measurement, simulation, and modeling. John Wiley & Sons, New York

    Google Scholar 

  61. Galante G, d. Bona LCE (2012) A survey on cloud computing elasticity, In: Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing, ser. UCC ’12. Washington, DC, USA: IEEE Computer Society, pp 263–270. [Online]. Available: https://doi.org/10.1109/UCC.2012.30

  62. Iqbal W, Dailey M, Carrera D (2009) SLA-driven adaptive resource management for web applications on a heterogeneous compute cloud. Springer Verlag, pp 243–253. [Online]. Available: http://hdl.handle.net/2117/15867

  63. Iqbal W, Erradi A, Abdullah M, Mahmood A (2022) Predictive auto-scaling of multi-tier applications using performance varying cloud resources. IEEE Trans Cloud Comput 10(1):595–607

    Article  Google Scholar 

  64. Qu C, Calheiros RN, Buyya R (2018) Auto-scaling web applications in clouds: A taxonomy and survey, ACM Comput Surv, 51(4). [Online]. Available: https://doi.org/10.1145/3148149

  65. Gotin M, Lösch F, Heinrich R, Reussner R (2018) Investigating performance metrics for scaling microservices in cloudiot-environments, In: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering, ser. ICPE ’18. Association for Computing Machinery, New York, NY, USA, pp 157–167. [Online]. Available: https://doi.org/10.1145/3184407.3184430

  66. Tang B (1993) Orthogonal array-based Latin hypercubes. J Am Stat Assoc 88(424):1392–1397

    Article  MathSciNet  MATH  Google Scholar 

  67. Mishra N, Lafferty JD, Hoffmann H (2017) Esp: A machine learning approach to predicting application interference. In: IEEE International Conference on Autonomic Computing (ICAC), pp 125–134

  68. Karson M (1968) Handbook of methods of applied statistics. volume i: Techniques of computation descriptive methods, and statistical inference. volume ii: Planning of surveys and experiments. i. m. chakravarti, r. g. laha, and j. roy, new york, john wiley; 1967, \$9.00. J Am Stat Assoc 63(323):1047–1049

    Google Scholar 

  69. Efron B, Tibshirani R (1993) An introduction to the bootstrap. Chapman and Hall, London

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centro de Informática (CIn), Federal University of Pernambuco, Road Jorn. Aníbal Fernandes - Cidade Universitária, Recife, PE, 50740-560, Brazil

    Thiago Felipe da Silva Pinheiro, Paulo Pereira & Paulo Maciel

  2. Research, Microsoft, Redmond Washington, USA

    Bruno Silva

Authors
  1. Thiago Felipe da Silva Pinheiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paulo Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo Maciel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thiago Felipe da Silva Pinheiro.

Ethics declarations

Conflict of interest

The authors have no conflict of interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Silva Pinheiro, T.F., Pereira, P., Silva, B. et al. A performance modeling framework for microservices-based cloud infrastructures. J Supercomput 79, 7762–7803 (2023). https://doi.org/10.1007/s11227-022-04967-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04967-6

Keywords

Search

Navigation