Abstract
We extend the population continuous time Markov chain formalism so that the state space is augmented with continuous variables accumulated over time as functions of component populations. System feedback can be expressed using accumulations that in turn can influence the Markov chain behaviour via functional transition rates. We show how to obtain mean-field differential equations capturing means and higher-order moments of the discrete populations and continuous accumulation variables. We also provide first- and second-order convergence results and suggest a novel normal moment closure that can greatly improve the accuracy of means and higher moments.
We demonstrate how such a framework is suitable for modelling feedback from globally-accumulated quantities such as energy consumption, cost or temperature. Finally, we present a worked example modelling a hypothetical heterogeneous computing cluster and its interaction with air conditioning units.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Billingsley, P.: Convergence of Probability Measures. John Wiley & Sons (1968)
Bortolussi, L.: Hybrid Limits of Continuous Time Markov Chains. In: QEST 2011, pp. 3–12. IEEE, Aachen (2011)
Bortolussi, L., Galpin, V., Hillston, J.: HYPE with stochastic events. EPTCS 57, 120–133 (2011)
Cain, M.: The moment-generating function of the minimum of bivariate normal random variables. The American Statistician 48(2), 124–125 (1994)
Davis, M.H.A.: Markov models and optimization. Chapman & Hall/CRC (1993)
Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley (2005)
Gast, N., Bruno, G.: A mean field model of work stealing in large-scale systems. In: SIGMETRICS, p. 13. ACM, New York (2010)
Gast, N., Gaujal, B.: Mean field limit of non-smooth systems and differential inclusions. SIGMETRICS Performance Evaluation Review 38(2), 30–32 (2010)
Gillespie, C.S.: Moment-closure approximations for mass-action models. IET Systems Biology 3(1), 52–58 (2009)
Hayden, R.A., Bradley, J.T.: A fluid analysis framework for a Markovian process algebra. Theoretical Computer Science 411(22-24), 2260–2297 (2010)
Hayden, R.A., Stefanek, A., Bradley, J.T.: Fluid computation of passage-time distributions in large Markov models. Theoretical Computer Science 413(1), 106–141 (2012)
Hillston, J.: Fluid flow approximation of PEPA models. In: QEST, pp. 33–42. IEEE (September 2005)
Horton, G., Kulkarni, V.G., Nicol, D.M., Trivedi, K.S.: Fluid stochastic Petri nets: Theory, applications, and solution techniques. European Journal of Operational Research 105(1), 184–201 (1998)
Kallenberg, O.: Foundations of Modern Probability. Springer (2002)
Khadim, U.: A comparative study of process algebras for hybrid systems. Computer Science Report 06–23, Technische Universiteit Eindhoven (2006)
Klebaner, F.C.: Introduction to stochastic calculus with applications, 2nd edn. Imperial College Press (2006)
Rawson, A., Pfleuger, J., Cader, T.: Data Center Power Efficiency Metrics: PUE and DCiE. The Green Grid (2007)
de Souza e Silva, E., Gail, R.: An algorithm to calculate transient distributions of cummulative rate and impulse-based rewards. Communications in Statistics: Stochastic Models 14(3), 509–536 (1998)
Silva, M., Júlvez, J., Mahulea, C., Vázquez, C.R.: On fluidization of discrete event models: observation and control of continuous Petri nets. Discrete Event Dynamic Systems 21(4), 427–497 (2011)
Stefanek, A., Hayden, R.A., Bradley, J.T.: A new tool for the performance analysis of massively parallel computer systems. In: QAPL. Electronic Proceedings in Theoretical Computer Science (March 2010)
Stefanek, A., Hayden, R.A., Bradley, J.T.: Fluid Analysis of Energy Consumption using Rewards in Massively Parallel Markov Models. In: Computing, p. 121. ACM Press (2011)
Stefanek, A., Hayden, R.A., Bradley, J.T.: GPA – A Tool for Fluid Scalability Analysis of Massively Parallel Systems. In: QEST, pp. 147–148. IEEE (September 2011)
Tang, Q., Gupta, S., Varsamopoulos, G.: Energy-efficient thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach. IEEE Transactions on Parallel and Distributed Systems 19(11), 1458–1472 (2008)
Telek, M., Rácz, S.: Numerical analysis of large Markovian reward models. Performance Evaluation, 36–37, 95–114 (August 1999)
Tribastone, M., Gilmore, S., Hillston, J.: Scalable Differential Analysis of Process Algebra Models. IEEE Transactions on Software Engineering 38(1), 205–219 (2012)
Whitt, W.: Internet supplement to Stochastic-Process Limits (2002), http://www.columbia.edu/~ww2040/supplement.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stefanek, A., Hayden, R.A., Mac Gonagle, M., Bradley, J.T. (2012). Mean-Field Analysis of Markov Models with Reward Feedback. In: Al-Begain, K., Fiems, D., Vincent, JM. (eds) Analytical and Stochastic Modeling Techniques and Applications. ASMTA 2012. Lecture Notes in Computer Science, vol 7314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30782-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-30782-9_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30781-2
Online ISBN: 978-3-642-30782-9
eBook Packages: Computer ScienceComputer Science (R0)