Abstract
Model-based management of software applications in the cloud is based on predicted delays at scaled out services. These services are modeled as FIFO (first-in first-out) multiservers, with many servers, users and types of operation (classes of service). Efficient approximations for these multiservers either scale badly for large systems, or have convergence and accuracy problems. This work investigates three scalable approximations in depth. The best (called AB) combines class aggregation and a binomial approximation to the queue state (which assumes that users behave independently). Over the parameters of greatest relevance, two-thirds of the errors are less than 5%. The largest errors, up to about 30%, occur near the onset of saturation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akyildiz, I.F., Bolch, G.: Mean value analysis approximation for multiple server queueing networks. Perform. Eval. 8, 77–91 (1988)
Bolch, G., Greiner, S., Meer, H., Trivedi, K.: Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications, 2nd edn. Wiley, Hoboken (2006)
Casale, G., Perez, J., Wang, W.: QD-AMVA: evaluating systems with queue-dependent service requirements. Perform. Eval. 91, 80–98 (2015)
Casale, G.: Integrated performance evaluation of extended queueing network models with line. In: Proceedings of the Winter Simulation Conference, pp. 2377–2388 (2020)
Chandy, K.M., Neuse, D.: Linearizer: a heuristic algorithm for queueing network models of computing systems. Comm. of the ACM 25(2), 126–134 (1982)
Conway, A.E.: Fast approximate solution of queueing networks with multi-server chain-dependent FCFS queues. In: Modeling Techniques and Tools for Computer Performance Evaluation, Plenum, New York, pp 385–396 (1989). https://doi.org/10.1007/978-1-4613-0533-0_25
Dowdy, L.W., Carlson, B.M., Krantz, A.T., Tripathi, S.K.: Single-class bounds of multi-class queuing networks. J. ACM 39(1), 188–213 (1992)
Franks, G.: Performance analysis of distributed server systems. Ph.D thesis, Carleton University (1999)
G. Franks, G.: lqngen − generate layered queueing network models. https://github.com/layeredqueuing/V5. Accessed 10 Feb 2022
Franks, G., Al-Omari, T., Woodside, M., Das, O., Derisavi, S.: Enhanced modeling and solution of layered queueing networks. IEEE Trans. Software Engineering 35(2), 148–161 (2009)
Franks, G. et al: Layered Queueing Network Solver and Simulator User Manual, Carleton University. http://www.sce.carleton.ca/rads/lqns/userman22.pdf. Accessed 20 Jan 2022
Gias, A.U., Casale, G., Woodside, M.: ATOM: model-driven autoscaling for microservices. In: 39th International Conference on Distributed Computing Systems (ICDCS), pp 1994–2004 (2019)
Legato, P., Mazza, R.M.: Class aggregation for multi-class queueing networks with FCFS multi-server stations. In: Phung-Duc, T., Kasahara, S., Wittevrongel, S. (eds.) QTNA 2019. LNCS, vol. 11688, pp. 221–239. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27181-7_14
Reiser, M., Lavenberg, S.S.: Mean-value analysis of closed multichain queueing networks. J. ACM 27(2), 312–322 (1980)
Rolia, J.A., Sevcik, K.C.: The method of layers. IEEE Trans. Softw. Eng. 21, 689–700 (2015)
Ruth, A.: Entwicklung, Implementierung und Validierung neuer Approximationsverfahren fur die Mittelwertanalyse (MWA) zur Leistungsberechnung von Rechnersystemen. Diplomarbeit am IMMD der Friedrich-Alexander-Universitat Erlangen-Nurnberg (1987)
Schmidt, R.: An approximate MVA algorithm for exponential, class-dependent multiple servers. Perform. Eval. 29, 245–254 (1997)
Schweitzer, P.J.: Approximate analysis of multiclass closed networks of queues. In: Proceedings of the International Conference on Stochastic Control and Optimization, Amsterdam, pp 25–29 (1979)
Silva, E.D.S., Muntz, R.R.: Approximate solutions for a class of non-product form queueing network models. Perform. Eval. 7, 221–242 (1987)
Zhang, Q., Xiao, Y., Liu, F., Lui, J.C.S., Guo, J., Wang, T.: Joint optimization of chain placement and request scheduling for network function virtualization. In: International Conference on Distributed Computing Systems (ICDCS), pp 731–741 (2011)
Zhang, Q., Zhu, Q., Zhani, M.F., Boutaba, R., Hellerstein, J.L.: Dynamic service placement in geographically distributed clouds. IEEE J. Sel. Areas Commun. 31, 762–772 (2013)
Zhou, S.: A New Approximation for Multiserver Waiting Time for Layered Queueing Systems. MASc thesis, Carleton University (2021)
Zhou, S., Woodside, M.: A multiserver approximation for cloud scaling analysis. In: Proceedings of the Workshop on Challenges in Software Performance (WOSPC-22), in the Companion Volume to the International Conference on Performance Engineering (ICPE22), ACM, New York (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, S., Woodside, M. (2023). A Robust Approximation for Multiclass Multiserver Queues with Applications to Microservices Systems. In: Gilly, K., Thomas, N. (eds) Computer Performance Engineering. EPEW 2022. Lecture Notes in Computer Science, vol 13659. Springer, Cham. https://doi.org/10.1007/978-3-031-25049-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-25049-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25048-4
Online ISBN: 978-3-031-25049-1
eBook Packages: Computer ScienceComputer Science (R0)