Skip to main content
SpringerLink
Log in
Book cover

International Conference on Quantitative Evaluation of Systems

QEST 2021: Quantitative Evaluation of Systems pp 310–328Cite as

Service Demand Distribution Estimation for Microservices Using Markovian Arrival Processes

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12846))

Abstract

Building performance models for microservices applications in DevOps is costly and error-prone. Accurate service demand distribution estimation is critical to performance model parameterization. However, traditional service demand estimation methods focus on capturing the mean service demand, disregarding higher-order moments of the distribution. To address this limitation, we propose to estimate higher moments of the service demand distribution for a microservice from monitoring traces. We first generate a closed queueing model to abstract a microservice and model the departure process at the queue node as a Markovian arrival process. This allows formulating the estimation of service demand as an optimization problem, which aims to find the optimal parameters of the first multiple moments of the service demand distribution based on the inter-departure times. We then estimate the service demand distribution with a novel maximum likelihood algorithm, and heuristics to mitigate the computational cost of the optimization process for scalability. We apply our method to real traces from a microservice-based application and demonstrate that its estimations lead to greater prediction accuracy than exponential distributions assumed in traditional service demand estimation approaches.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/go-chassis/go-bmi.

  2. 2.

    https://microservices-demo.github.io/.

  3. 3.

    https://locust.io/.

References

  1. Andersen, A.T., Neuts, M.F., Nielsen, B.F.: Lower order moments of inter-transition times in the stationary QBD process. Methodol. Comput. Appl. Probab. 2(4), 339–357 (2000)

    Article  MathSciNet  Google Scholar 

  2. Bass, L., Weber, I., Zhu, L.: DevOps: A Software Architect’s Perspective. Addison-Wesley Professional, Boston (2015)

    Google Scholar 

  3. Bezemer, C.P., et al.: How is performance addressed in devops?. In: Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering, pp. 45–50 (2019)

    Google Scholar 

  4. Binz, T., Breitenbücher, U., Kopp, O., Leymann, F.: TOSCA: portable automated deployment and management of cloud applications. In: Bouguettaya, A., Sheng, Q., Daniel, F. (eds.) Advanced Web Services. Springer, New York (2014). https://doi.org/10.1007/978-1-4614-7535-4_22

  5. Bobbio, A., Horváth, A., Telek, M.: Matching three moments with minimal acyclic phase type distributions. Stoch. Models 21(2–3), 303–326 (2005)

    Article  MathSciNet  Google Scholar 

  6. Bramson, M., Lu, Y., Prabhakar, B.: Randomized load balancing with general service time distributions. ACM SIGMETRICS Perform. Eval. Rev. 38(1), 275–286 (2010)

    Article  Google Scholar 

  7. Brosig, F., Kounev, S., Krogmann, K.: Automated extraction of palladio component models from running enterprise java applications. In: Proceedings of the Fourth International ICST Conference on Performance Evaluation Methodologies and Tools, pp. 1–10 (2009)

    Google Scholar 

  8. Casale, G., et al.: Radon: rational decomposition and orchestration for serverless computing. SICS Softw.-Intensiv. Cyber-Phys. Syst. 35(1), 77–87 (2020)

    Google Scholar 

  9. Casale, G., Cremonesi, P., Turrin, R.: How to select significant workloads in performance models. In: CMG Conference Proceedings, pp. 58–108. Citeseer (2007)

    Google Scholar 

  10. Casale, G., Cremonesi, P., Turrin, R.: Robust workload estimation in queueing network performance models. In: 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2008), pp. 183–187. IEEE (2008)

    Google Scholar 

  11. Dudin, S., Dudina, O.: Retrial multi-server queuing system with PHF service time distribution as a model of a channel with unreliable transmission of information. Appl. Math. Model. 65, 676–695 (2019)

    Article  MathSciNet  Google Scholar 

  12. Duggan, M., Mason, K., Duggan, J., Howley, E., Barrett, E.: Predicting host CPU utilization in cloud computing using recurrent neural networks. In: 2017 12th International Conference for Internet Technology and Secured Transactions (ICITST), pp. 67–72. IEEE (2017)

    Google Scholar 

  13. Franks, G., Al-Omari, T., Woodside, M., Das, O., Derisavi, S.: Enhanced modeling and solution of layered queueing networks. IEEE Trans. Softw. Eng. 35(2), 148–161 (2008)

    Article  Google Scholar 

  14. Garetto, M., Cigno, R.L., Meo, M., Marsan, M.A.: A detailed and accurate closed queueing network model of many interacting TCP flows. In: Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No. 01CH37213), vol. 3, pp. 1706–1715. IEEE (2001)

    Google Scholar 

  15. Horváth, A., Telek, M.: PhFit: a general phase-type fitting tool. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 82–91. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46029-2_5

    Chapter  Google Scholar 

  16. Horváth, G., Telek, M.: BuTools 2: a rich toolbox for Markovian performance evaluation. In: VALUETOOLS (2016)

    Google Scholar 

  17. Krishna, K., Murty, M.N.: Genetic k-means algorithm. IEEE Trans. Syst. Man Cybern., Part B (Cybern.) 29(3), 433–439 (1999)

    Article  Google Scholar 

  18. Krishnamoorthy, A., Manikandan, R., Lakshmy, B.: A revisit to queueing-inventory system with positive service time. Ann. Oper. Res. 233(1), 221–236 (2015)

    Article  MathSciNet  Google Scholar 

  19. Liao, S., Zhang, H., Shu, G., Li, J.: Adaptive resource prediction in the cloud using linear stacking model. In: 2017 Fifth International Conference on Advanced Cloud and Big Data (CBD), pp. 33–38. IEEE (2017)

    Google Scholar 

  20. Marsan, M.A., Balbo, G., Conte, G., Donatelli, S., Franceschinis, G.: Modelling with Generalized Stochastic Petri Nets, vol. 292. Wiley, New York (1995)

    MATH  Google Scholar 

  21. Mazkatli, M., Koziolek, A.: Continuous integration of performance model. In: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering, pp. 153–158 (2018)

    Google Scholar 

  22. Neuts, M.F.: Models based on the Markovian arrival process. IEICE Trans. Commun. 75(12), 1255–1265 (1992)

    Google Scholar 

  23. O’Cinneide, C.A.: Characterization of phase-type distributions. Stoch. Models 6(1), 1–57 (1990)

    Article  MathSciNet  Google Scholar 

  24. Pan, J.X., Fang, K.T.: Maximum likelihood estimation. In: Growth Curve Models and Statistical Diagnostics. Springer Series in Statistics. Springer, New York (2002). https://doi.org/10.1007/978-0-387-21812-0_3

  25. Parini, A., Pattavina, A.: Modelling voice call interarrival and holding time distributions in mobile networks. In: Proceedings of ITC 2005, pp. 729–738 (2005)

    Google Scholar 

  26. Pérez, J.F., Pacheco-Sanchez, S., Casale, G.: An offline demand estimation method for multi-threaded applications. In: 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems, pp. 21–30. IEEE (2013)

    Google Scholar 

  27. Pérez, J.F., Casale, G.: Line: evaluating software applications in unreliable environments. IEEE Trans. Reliab. 66(3), 837–853 (2017)

    Article  Google Scholar 

  28. Petriu, D.C., Shen, H.: Applying the UML performance profile: graph grammar-based derivation of LQN models from UML specifications. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 159–177. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46029-2_10

    Chapter  MATH  Google Scholar 

  29. Reibman, A., Trivedi, K.: Numerical transient analysis of Markov models. Comput. Oper. Res. 15(1), 19–36 (1988)

    Article  Google Scholar 

  30. Rolia, J., Vetland, V.: Correlating resource demand information with arm data for application services. In: Proceedings of the 1st International Workshop on Software and Performance, pp. 219–230 (1998)

    Google Scholar 

  31. Schweitzer, P.: Approximate analysis of multiclass closed networks of queues. J. ACM 29(2) (1981)

    Google Scholar 

  32. Sidje, R.B., Burrage, K., MacNamara, S.: Inexact uniformization method for computing transient distributions of Markov chains. SIAM J. Sci. Comput. 29(6), 2562–2580 (2007)

    Article  MathSciNet  Google Scholar 

  33. Spinner, S., Casale, G., Brosig, F., Kounev, S.: Evaluating approaches to resource demand estimation. Perform. Eval. 92, 51–71 (2015)

    Article  Google Scholar 

  34. Wang, W., Huang, X., Qin, X., Zhang, W., Wei, J., Zhong, H.: Application-level CPU consumption estimation: towards performance isolation of multi-tenancy web applications. In: 2012 IEEE Fifth International Conference on Cloud Computing, pp. 439–446. IEEE (2012)

    Google Scholar 

Download references

Acknowledgement

The work of Giuliano Casale has been partly funded by the EU’s Horizon 2020 program under grant agreement No. 825040.

Author information

Authors and Affiliations

  1. Department of Computing, Imperial College London, London, UK

    Runan Wang, Giuliano Casale & Antonio Filieri

Authors
  1. Runan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuliano Casale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Filieri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Runan Wang .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Alessandro Abate

  2. Ca’ Foscari University of Venice, Venice, Italy

    Andrea Marin

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, R., Casale, G., Filieri, A. (2021). Service Demand Distribution Estimation for Microservices Using Markovian Arrival Processes. In: Abate, A., Marin, A. (eds) Quantitative Evaluation of Systems. QEST 2021. Lecture Notes in Computer Science(), vol 12846. Springer, Cham. https://doi.org/10.1007/978-3-030-85172-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85172-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85171-2

  • Online ISBN: 978-3-030-85172-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation