Abstract
Software rejuvenation is a preventive and proactive maintenance solution that is particularly useful for counteracting the phenomenon of software aging. In this paper we consider an operational software system with multiple degradations and derive the optimal software rejuvenation policy minimizing the expected operation cost per unit time in the steady state, via the dynamic programing approach. Especially, we develop a reinforcement learning algorithm to estimate the optimal rejuvenation schedule adaptively and examine its asymptotic properties through a simulation experiment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abounadi, J., Bertsekas, D., Borkar, V.S.: Learning algorithms for Markov decision processes with average cost. SIAM J. Control and Optimization 40, 681–698 (2001)
Adams, E.: Optimizing preventive service of the software products. IBM J. Research & Development 28, 2–14 (1984)
Avritzer, A., Weyuker, E.J.: Monitoring smoothly degrading system for increased dependabulity. Empirical Software Eng. 2, 59–77 (1997)
Bertsekas, D.P., Tsitsiklis, N.J.: Neuro-Dynamic Programming. Atheena Scientific (1996)
Bobbio, A., Sereno, M., Anglano, C.: Fine grained software degradation models for optimal rejuvenation policies. Performance Evaluation 46, 45–62 (2001)
Borkar, V.S., Meyn, S.P.: The O.D.E method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control and Optimization 38, 447–469 (2000)
Castelli, V., Harper, R.E., Heidelberger, P., Hunter, S.W., Trivedi, K.S., Vaidyanathan, K.V., Zeggert, W.P.: Proactive management of software aging. IBM J. Research & Development 45, 311–332 (2001)
Dohi, T., Goševa-Popstojanova, K., Trivedi, K.S.: Estimating software rejuvenation schedule in high assurance systems. The Computer Journal 44, 473–485 (2001)
Dohi, T., Goševa-Popstojanova, K., Vaidyanathan, K.V., Trivedi, K.S., Osaki, S.: Software rejuvenation modeling and applications. In: Pham, H. (ed.) Handbook of Reliability Engineering, pp. 245–268. Springer, Heidelberg (2003)
Eto, H., Dohi, T.: Determining the optimal software rejuvenation schdule via semi-Markov decision process. J. Computer Science 2, 528–534 (2006)
Garg, S., Telek, M., Puliafito, A., Trivedi, K.S.: Analysis of software rejuvenation using Markov regenerative stochastic Petri net. In: Proc. 6th Intl Symp. on Software Reliab. Eng., pp. 24–27 (1995)
Garg, S., Pfening, S., Puliafito, A., Telek, M., Trivedi, K.S.: Analysis of preventive maintenance in transactions based software systems. IEEE Trans. on Computers 47, 96–107 (1998)
Gosavi, A.: Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning. Kluwer Academic Publishers, Dordrecht (2003)
Huang, Y., Kintala, C., Kolettis, N., Fulton, N.D.: Software rejuvenation: analysis, module and applications. In: Proc. 25th Intl Symp. on Fault Tolerant Computing, pp. 381–390 (1995)
Konda, V.R., Borkar, V.S.: Actor-critic-type learning algorithms for Markov decision processes. SIAM J. Control and Optimization 38, 94–123 (1999)
Mahadevan, S.: Average reward reinforcement learning: foundations, algorithms for Markov decision processes. SIAM J. Control and Optimization 38, 94–123 (2000)
Pfening, S., Garg, S., Puliafito, A., Telek, M., Trivedi, K.S.: Optimal rejuvenation for toleranting soft failure. Performance Evaluation 27/28, 491–506 (1996)
Sutton, R.S., Barto, A.: Reinforcement Learning. MIT Press, Cambridge (1998)
Suzuki, H., Dohi, T., Goševa-Popstojanova, K., Trivedi, K.S.: Analysis of multi step failure models with periodic software rejuvenation. In: Artalejo, J.R., Krishnamoorthy, A. (eds.) Advances in Stochastic Modelling, pp. 85–108. Notable Publications (2002)
Tijms, H.C.: Stochastic Models: An Algorithmic Approach. John Wiley & Sons, Chichester (1994)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Eto, H., Dohi, T., Ma, J. (2008). Simulation-Based Optimization Approach for Software Cost Model with Rejuvenation. In: Rong, C., Jaatun, M.G., Sandnes, F.E., Yang, L.T., Ma, J. (eds) Autonomic and Trusted Computing. ATC 2008. Lecture Notes in Computer Science, vol 5060. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69295-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-69295-9_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69294-2
Online ISBN: 978-3-540-69295-9
eBook Packages: Computer ScienceComputer Science (R0)