Abstract
Stock management in a hospital requires the achievement of a trade-off between conflicting criteria, with mandatory requirements on the quality of patient care as well as on purchasing and logistics costs. We address daily drug ordering in a ward of an Italian public hospital, where patient admission/discharge and drug consumption during the sojourn are subject to uncertainty. To derive optimal control policies minimizing the overall purchasing and stocking cost while avoiding drug shortages, the problem is modeled as a Markov Decision Process (MDP), fitting the statistics of hospitalization time and drug consumption through a discrete phase-type (DPH) distribution or a Hidden Markov Model (HMM). A planning algorithm that operates at run-time iteratively synthesizes and solves the MDP over a finite horizon, applies the first action of the best policy found, and then moves the horizon forward by one day. Experiments show the convenience of the proposed approach with respect to baseline inventory management policies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An Architectural Blueprint for Autonomic Computing. Technical report, IBM (2006)
Apache velocity (2018). http://velocity.apache.org/
HMMLearn python library (2018). http://hmmlearn.readthedocs.io/en/latest/
Project LINFA (2018). http://www.linfasystem.it/
Adachi, Y., Nose, T., Kuriyama, S.: Optimal inventory control policy subject to different selling prices of perishable commodities. Int. J. Prod. Econ. 60–61, 389–394 (1999)
Ahiska, S.S., Appaji, S.R., King, R.E., Warsing, D.P.: A Markov decision process-based policy characterization approach for a stochastic inventory control problem with unreliable sourcing. Int. J. Prod. Econ. 144(2), 485–496 (2013)
Alessandri, A., Gaggero, M., Tonelli, F.: Min-max and predictive control for the management of distribution in supply chains. IEEE Trans. Control Syst. Technol. 19(5), 1075–1089 (2011)
Blair, G., Bencomo, N., France, R.B.: Models@ run. time. Computer 42, 10 (2009)
Bobbio, A., Horváth, A., Scarpa, M., Telek, M.: Acyclic discrete phase type distributions: properties and a parameter estimation algorithm. Perform. Eval. 54(1), 1–32 (2003)
de Vries, J.: The shaping of inventory systems in health services: a stakeholder analysis. Int. J. Prod. Econ. 133(1), 60–69 (2011)
García, C.E., Prett, D.M., Morari, M.: model predictive control: theory and practice-a survey. Automatica 25(3), 335–348 (1989)
Giannoccaro, I., Pontrandolfo, P.: Inventory management in supply chains: a reinforcement learning approach. Int. J. Prod. Econ. 78(2), 153–161 (2002)
Horváth, A., Telek, M.: PhFit: a general phase-type fitting tool. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 82–91. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46029-2_5
Incerto, E., Tribastone, M., Trubiani, C.: Software performance self-adaptation through efficient model predictive control. In: ASE, Piscataway, NJ, USA, pp. 485–496. IEEE Press (2017)
Jurado, I., et al.: Stock management in hospital pharmacy using chance-constrained model predictive control. Comput. Biol. Med. 72, 248–255 (2016)
Kelle, P., Woosley, J., Schneider, H.: Pharmaceutical supply chain specifics and inventory solutions for a hospital case. Oper. Res. Health Care 1(2), 54–63 (2012)
Kwiatkowska, M., Norman, G., Parker, D.: PRISM: probabilistic symbolic model checker. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 200–204. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46029-2_13
Moreno, G.A., Cámara, J., Garlan, D., Schmerl, B.: Proactive self-adaptation under uncertainty: a probabilistic model checking approach. In: Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, pp. 1–12. ACM (2015)
Neuts, M.F.: Probability Distributions of Phase Type. Purdue University, Baltimore (1974). Department of Statistics
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2014)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)
Tempelmeier, H.: Inventory Management in Supply Networks Problems, Models, Solutions. Books on Demand, Norderstedt (2011)
Volland, J., Fügener, A., Schoenfelder, J., Brunner, J.O.: Material logistics in hospitals: a literature review. Omega 69, 82–101 (2017)
White, D.J.: A survey of applications of markov decision processes. J. Oper. Res. Soc. 44(11), 1073–1096 (1993)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Biagi, M., Carnevali, L., Santoni, F., Vicario, E. (2018). Hospital Inventory Management Through Markov Decision Processes @runtime. In: McIver, A., Horvath, A. (eds) Quantitative Evaluation of Systems. QEST 2018. Lecture Notes in Computer Science(), vol 11024. Springer, Cham. https://doi.org/10.1007/978-3-319-99154-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-99154-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99153-5
Online ISBN: 978-3-319-99154-2
eBook Packages: Computer ScienceComputer Science (R0)