Cited By
View all- Mahawish AHassan H(2022)Improving RED algorithm congestion control by using the Markov decision processScientific Reports10.1038/s41598-022-17528-x12:1Online publication date: 3-Aug-2022
Network Risks in Markov Decision Processes
Pages 7 - 10
Abstract
A class of Markov Decision Making Processes (MDP) is proposed in this work, considering the network risks. Risk is considered as a product of two measures one of which is the probability for an adverse event at the process' passing through a given state. It is proved that in case of the same values of these probabilities a network flow of risks is received which has one-to-one mapping to the MDP. Relations between these two controllable processes are obtained.
A case is investigated when the probabilities of adverse events are different for the different states and a method is proposed through which in this case the optimal solutions for the MDP with risks can be found. The results received are confirmed by appropriate numerical examples.
The possible areas of application of the MDP with risks being proposed are pointed out.
References
[1]
Sgurev, V, Markov Flows, Sofia, Publishing House of Bulg. Acad. Of Sci., 1993, ISBN 954-430-103-8, https://search.rsl.ru/ru/record/01000556731
[2]
Sgurev V., St. Drangajov - Intelligent Control of Flows with Risks on a Network, Proceedings of the 7th IEEE International Conference Intelligent Systems - IS'14, September 24-26 2014, Warsaw, Poland, ISSN 2194-5357, ISSN 2194-5365 (electronic), ISBN 978-3-319-11309-8, ISBN 978-3-319-11310-4 (eBook), DOI 10.1007/978-3-319-11310-4, Volume 2: Tools, Architectures, Systems, Applications, Springer International Publishing, Switzerland, P. Angelov et al. (eds.), Advances in Intelligent Systems and Computing vol. 323, 2014, pp. 27--35.
[3]
D. R. Ford, D. R. Fulkerson, Flows in Networks, Princeton University Press Princeton, NJ, USA © 2010, ISBN:0691146675 9780691146676, https://dl.acm.org/citation.cfm?id=1942094
[4]
Christofides N., Graph theory: An Algorithmic Approach, London [etc.], Academic Press, 1986.
[5]
H. Mine and S. Osaki, Markovian Decision Processes, Copyright © 1971 Society for Industrial and Applied MathematicsRead, ISSN (print): 0036-1445, ISSN (print): 0036-1445
[6]
Eitan Altman, Constrained Markov Decision Processes, Chapman and Hall/CRC Published March 30, 1999, ISBN 9780849303821 - CAT# C0382
[7]
Sgurev, V., Network Flows with General Constraints, Publishing House of the Bulgarian Academy of Sciences, Sofia, 1991 (in Bulgarian)
[8]
Ruszczyński, Risk-averse dynamic programming for Markov decision processes, © Springer and Mathematical Optimization Society 2010, A. Math. Program. (2010) 125: 235. https://doi.org/10.1007/s10107-010-0393-3
[9]
Ravindra K. Ahuja, Thomas L. Magnanti, James B. Orlin, Network Flows: Theory, Algorithms, and Applications, Pearson, 2013, ISBN 1292042702, 9781292042701
[10]
P. A. Jensen, W. J. P. Barnes, Network Flow Programming, Krieger Pub Co, 1987, ISBN-13:978--0894642104, ISBN-10:0894642103
Index Terms
- Network Risks in Markov Decision Processes
Recommendations
Solving Markov decision processes with downside risk adjustment
Markov decision processes (MDPs) and their variants are widely studied in the theory of controls for stochastic discreteevent systems driven by Markov chains. Much of the literature focusses on the risk-neutral criterion in which the expected rewards, ...
Variability Sensitive Markov Decision Processes
Considered are time-average Markov Decision Processes MDPs with finite state and action spaces. Two definitions of variability are introduced, namely, the expected time-average variability and time-average expected variability. The two criteria are in ...
Variance-Penalized Markov Decision Processes
We consider a Markov decision process with both the expected limiting average, and the discounted total return criteria, appropriately modified to include a penalty for the variability in the stream of rewards. In both cases we formulate appropriate ...
Comments
Information & Contributors
Information
Published In
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- UORB: University of Ruse, Bulgaria
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 August 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CompSysTech '20
CompSysTech '20: International Conference on Computer Systems and Technologies '20
June 19 - 20, 2020
Ruse, Bulgaria
Acceptance Rates
CompSysTech '20 Paper Acceptance Rate 46 of 72 submissions, 64%;
Overall Acceptance Rate 241 of 492 submissions, 49%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 23Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
Cited By
View all- Mahawish AHassan H(2022)Improving RED algorithm congestion control by using the Markov decision processScientific Reports10.1038/s41598-022-17528-x12:1Online publication date: 3-Aug-2022
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in