Abstract
We consider a hybrid extension of population continuous time Markov chains (PCTMC)—a class of Markov processes capturing interactions between large groups of identically behaved agents. We augment the discrete state space of a PCTMC with continuous variables that evolve as integrals over the population vector and that can simultaneously provide feedback to the rates of transitions in the PCTMC. Additionally, we include time-inhomogeneous rate parameters, which can be used to incorporate real measurement data into the models. We extend mean-field techniques for PCTMCs and show how to derive a system of integral equations that approximate the evolution of means and higher-order moments of populations and continuous variables in a hybrid PCTMC. We prove first- and second-order convergence results that justify the approximations. We use a moment closure based on the normal distribution which improves the accuracy of the moment approximation in case of proportional control where transition rates depend on the amount a continuous variable is above or below a fixed threshold. We demonstrate how this framework is suitable for modelling feedback from globally-accumulated quantities in a large scale system, such as energy consumption, total cost or temperature in a data centre. We present a model of a many server system with temperature management and external workload that varies with time. We show how to use real data to represent the workload within the framework. We use stochastic simulation to validate the example and an earlier example of a hypothetical heterogeneous computing cluster.
Similar content being viewed by others
Notes
Assuming that \(\mathbb {X}\) is finite then the arguments of Sect. 2 guarantee that, over finite intervals of time, the process \((\varvec{X}(t),\varvec{\mathcal {Y}}(t))\) is bounded to remain in some compact set, and then the boundedness requirement for the functions \(h\) need only be honoured on this set.
Note that it is then only strictly necessary for the theorem below that \(\varvec{f}, \,f_c\) and \(\varvec{g}\) are defined on \(\mathbb {S} \times [0,t_f]\) rather than on the whole of \(\mathbb {R}^{N+M} \times \mathbb {R}_+\).
Informally, this is ‘uniform convergence in distribution over \([0,t_f]\)’.
References
Anderson, D. F. (2007). A modified next reaction method for simulating chemical systems with time dependent propensities and delays. The Journal of chemical physics, 127(21), 214,107. doi:10.1063/1.2799998.
Bakhshi, R., Endrullis, J., Endrullis, S., Fokkink, W., & Haverkort, B. (2010). Automating the mean-field method for large dynamic gossip networks. International Conference on Quantitative Evaluation of Systems pp. 241–250. doi:10.1109/QEST.2010.38.
Billingsley, P. (1968). Convergence of probability measures. New York: Wiley.
Bortolussi, L. (2012). Hybrid behaviour of Markov population models. http://arxiv.org/abs/1211.1643.
Bortolussi, L., Galpin, V., & Hillston, J. (2011). HYPE with stochastic events. Proceedings in Theoretical Computer Science, 57, 120–133. doi:10.4204/EPTCS.57.9.
Bortolussi, L., & Hayden, R. A. (2013). Bounds on the deviation of discrete-time markov chains from their mean-field model. Performance Evaluation, 70(10), 736–749. doi:10.1016/j.peva.2013.08.012.
Cain, M. (1994). The moment-generating function of the minimum of bivariate normal random variables. The American Statistician, 48(2), 124–125. doi:10.1080/00031305.1994.10476039.
Chaintreau, A., Le Boudec, J. Y., & Ristanovic, N. (2009). The age of gossip: Spatial mean field regime. Performance Evaluation Review, 37(1), 109–120. doi:10.1145/2492101.1555363.
Coddington, E. A., & Levinson, N. (1955). Theory of ordinary differential equations. New York: McGraw-Hill Book Company.
Davis, M. H. A. (1993). Markov models and optimization. Boca Raton: Chapman & Hall.
de Souzae Silva, E., & Gail, R. (1998). An algorithm to calculate transient distributions of cumulative rate and impulse based reward. Communications in Statistics, 14(3), 509–536. doi:10.1080/15326349808807486.
Ethier, S. N., & Kurtz, T. G. (2005). Markov processes: Characterization and convergence. New York: Wiley.
Gast, N., & Bruno, G. (2010). A mean field model of work stealing in large-scale systems. In: SIGMETRICS, vol. 38, p. 13. ACM Press, New York (2010). doi:10.1145/1811039.1811042.
Gillespie, C. S. (2009). Moment-closure approximations for mass-action models. IET Systems Biology, 3(1), 52–58.
Gomez-Gardenes, J., Reinares, I., Arenas, A., & Floria, L. M. (2012). Evolution of cooperation in multiplex networks. Scientific Reports, 2, 620. doi:10.1038/srep00620.
Guenther, M.C., Stefanek, A., & Bradley, J.T. (2012). Moment closures for performance models with highly non-linear rates. In: Computer Performance Engineering 9th European Workshop, EPEW 2012, Munich, Germany, July 30, 2012 (pp. 32–47). Munich: Springer. doi:10.1007/978-3-642-36781-6_3.
Hasenauer, J., Wolf, V., Kazeroonian, A., & Theis, F.J. (2013). Method of conditional moments (MCM) for the Chemical easter equation: A unified framework for the method of moments and hybrid stochastic-deterministic models. Journal of Mathematical Biology. doi:10.1007/s00285-013-0711-5. http://www.ncbi.nlm.nih.gov/pubmed/23918091,
Hayden, R. A., & Bradley, J. T. (2010). A fluid analysis framework for a Markovian process algebra. Theoretical Computer Science, 411(22–24), 2260–2297. doi:10.1016/j.tcs.2010.02.001.
Hayden, R. A., Bradley, J. T., & Clark, A. (2013). Performance specification and evaluation with unified stochastic probes and fluid analysis. IEEE Transactions on Software Engineering, 39(1), 97–118. doi:10.1109/TSE.2012.1.
Hayden, R. A., Stefanek, A., & Bradley, J. T. (2012). Fluid computation of passage-time distributions in large Markov models. Theoretical Computer Science, 413(1), 106–141. doi:10.1016/j.tcs.2011.07.017.
Hillston, J. (2005). Fluid flow approximation of PEPA models. In: QEST, pp. 33–42. doi:10.1109/QEST.2005.12.
Horton, G., Kulkarni, V. G., Nicol, D. M., & Trivedi, K. S. (1998). Fluid stochastic Petri nets: Theory, applications, and solution techniques. European Journal of Operational Research, 105(1), 184–201. doi:10.1016/S0377-2217(97)00028-3.
Kallenberg, O. (2002). Foundations of modern probability. Berlin: Springer.
Khadim, U. (2006). A comparative study of process algebras for hybrid systems. Computer Science Report 06–23, Technische Universiteit Eindhoven.
Klebaner, F. C. (2006). Introduction to stochastic calculus with applications (2nd ed.). London: Imperial College Press.
Lewis, T. G. (2009). Network science: Theory and applications. London: Wiley.
Liu, Z., Chen, Y., Bash, C., Wierman, A., Gmach, D., Wang, Z., et al. (2012). Renewable and cooling aware workload management for sustainable data centers. ACM SIGMETRICS Performance Evaluation Review, 40(1), 175. doi:10.1145/2318857.2254779.
Martin, A. (2000). Workload characterization of the 1998 World Cup Web Site. Tech. Rep. 3. IEEE Network. doi:10.1109/65.844498.
Noël, P. A., Brummitt, C. D., & D’Souza, R. M. (2013). Controlling self-organizing dynamics on networks using models that self-organize. Physical Review Letters, 111, 078701. doi:10.1103/PhysRevLett.111.078701.
Rawson, A., Pfleuger, J., & Cader, T. (2007). Data Center Power Efficiency Metrics: PUE and DCiE. The Green Grid (2007).
Silva, M., Júlvez, J., Mahulea, C., & Vázquez, C. R. (2011). On fluidization of discrete event models: observation and control of continuous Petri nets. Discrete Event Dynamic Systems, 21(4), 427–497. doi:10.1007/s10626-011-0116-9.
Stefanek, A., Hayden, R. A., & Bradley, J. T. (2010). A new tool for the performance analysis of massively parallel computer systems. Electronic Proceedings in Theoretical Computer Science,. doi:10.4204/EPTCS.28.11.
Stefanek, A., Hayden, R.A., & Bradley, J.T. (2011). Fluid Analysis of Energy Consumption using Rewards in Massively Parallel Markov Models. In: Computing, p. 121. ACM Press (2011). doi:10.1145/1958746.1958767.
Stefanek, A., Hayden, R.A., & Bradley, J.T. (2011). GPA - A Tool for Fluid Scalability Analysis of Massively Parallel Systems. In: QEST, pp. 147–148. IEEE (2011). doi:10.1109/QEST.2011.26.
Stefanek, A., Hayden, R.A., Gonagle, M.M., & Bradley, J.T. (2012). Mean-Field Analysis of Markov Models with Reward Feedback. In: Analytical and Stochastic Modeling Techniques and Applications 19th International Conference, ASMTA 2012, Grenoble, France, June 4–6, 2012. Proceedings, pp. 193–211. Springer. doi:10.1007/978-3-642-30782-9_14.
Tang, Q., Gupta, S., & Varsamopoulos, G. (2008). Energy-efficient thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach. IEEE Transactions on Parallel and Distributed Systems, 19(11), 1458–1472.
Telek, M., & Rácz, S. (1999). Numerical analysis of large Markov reward models. Performance Evaluation, 36–37(1–4), 95–114. doi:10.1016/S0166-5316(99)00032-2.
Tribastone, M., Gilmore, S., & Hillston, J. (2012). Scalable differential analysis of process algebra models. IEEE Transactions on Software Engineering, 38(1), 205–219. doi:10.1109/TSE.2010.82.
Whitt, W. (2002). Internet supplement to Stochastic-Process Limits (2002). http://www.columbia.edu/~ww2040/supplement.html. Accessed 10 July 2013.
Acknowledgments
Anton Stefanek, Richard A. Hayden and Jeremy T. Bradley are funded by EPSRC on the Analysis of Massively Parallel Stochastic Systems (AMPS) Project (Rference EP/G011737/1).
Author information
Authors and Affiliations
Corresponding author
Appendix: Parameters used in the examples
Appendix: Parameters used in the examples
Values of rate and initial population parameters used in the client–server example, Figs. 2, 3, 4, 5 and 6:
\({r_{ data }}\) | \({r_{ task }}\) | \({r_{ reset }}\) | \({r_{ on }}\) | \({r_{ off }}\) | \({r_{ heat }}\) | \({r_{ cool }}\) | \({t_{ thresh }}\) | \({n_{{ C }}}\) | \({n_{{ S }}}\) | \({n_{{ A }}}\) |
---|---|---|---|---|---|---|---|---|---|---|
0.6 | 0.2 | 0.1 | 0.2 | 0.2 | 0.2 | 0.4 | 30 | 40 | 30 | 20 |
Values of rate and initial population parameters used in the worked example, Figures 13, 14, 15:
\({n_{{ C }}}\) | \({n_{{ S }}}\) | \({n_{{ A }}}\) | \({r_{q,1}}\) | \({r_{q,2}}\) | \({r_{s,1}}\) | \({r_{s,2}}\) | \({r_{ reset }}\) | \({r_{ wakeup }}\) | |
---|---|---|---|---|---|---|---|---|---|
20000 | 1000 | 100 | 0.2 | 0.5 | 0.2 | 0.2 | 0.2 | 0.3 |
\({r_{ on }}\) | \({r_{ off }}\) | \({t_0}\) | \({t_{ thresh }}\) | \({t_{ sleep }}\) | \({p_{A,s}}\) | \({p_{A,{ sl }}}\) | \({p_{A,1}}\) | \({p_{A,2}}\) | \({r_{ cool }}\) |
---|---|---|---|---|---|---|---|---|---|
0.2 | 0.2 | 25 | 20 | 23 | 10 | 1 | 30 | 37.5 | 0.026 |
The constants \(p_{B,\cdot }\) are set as the respective \(p_{A,\cdot }\) constants multiplied by \(1.7\) and the heat constants \(c_{\cdot , \cdot }\) are set as the corresponding \(p_{\cdot ,\cdot }\) constants multiplied by a conversion factor \(7.71\times 10^{-6}\).
Values of rate and initial population parameters used in the time-inhomogeneous worked example, Figures 17, 19, 20
\({n_{{ S }}}\) | \({n_{{ A }}}\) | \({\mu }\) | \({r_{ up }}\) | \({r_{ down }}\) | \({r_{ fail }}\) | \({r_{ on }}\) | \({r_{ off }}\) | \({t_0}\) | \({t_{ thresh }}\) | \({t_{ fail }}\) |
---|---|---|---|---|---|---|---|---|---|---|
150 | 50 | 50.0 | 10.0 | 5.0 | 1.0 | 2.0 | 0.5 | 25 | 25 | 35 |
\({p_{S}}\) | \({p_{{ idle }}}\) | \({p_{A}}\) | \({h_{ cool }}\) | \({h_S}\) | \({h_{ idle }}\) | |||||
---|---|---|---|---|---|---|---|---|---|---|
1.0 | 0.2 | 0.2 | 5.0 | 1.0 | 0.1 |
We obtained the arrival rate \(\lambda (t)\) from the World Cup 98 data (Arlitt 2000) available at http://ita.ee.lbl.gov/html/contrib/WorldCup.html. For illustration purposes, we have rescaled the arrival rate by a factor of \(10^{-5}\).
Rights and permissions
About this article
Cite this article
Stefanek, A., Hayden, R.A. & Bradley, J.T. Mean-field analysis of hybrid Markov population models with time-inhomogeneous rates. Ann Oper Res 239, 667–693 (2016). https://doi.org/10.1007/s10479-014-1664-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-014-1664-9