Skip to main content
SpringerLink
Log in

Semi-Markov Decision Processes with Vector Pay-Offs

  • Conference paper
  • First Online:
Proceedings of the Seventh International Conference on Mathematics and Computing

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1412))

  • 575 Accesses

Abstract

Semi-Markov decision processes with vector rewards are studied here under discounted pay-off criterion. The notion of optimality is replaced by Pareto optimality here. We show that a pure and stationary Pareto-optimal strategy exists along with the existence of an algorithm to construct an approximate version of Pareto curves, i.e., \(\epsilon \)-approximate Pareto curve in polynomial time as in a multi-objective linear programming problem. Semi-Markov decision processes with multiple objectives find applications in situations like dynamic goal programming where the decision-maker has more than one objective to be optimized simultaneously. Further, we also investigate the Pareto-realizability problem as well as the NP-completeness of pure stationary Pareto-realizability problem.

Supported by Department of Science and Technology, Govt. of India, INSPIRE Fellowship Scheme

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Jewell WS (1963) Markov-renewal programming. I: formulation, finite return models. Oper Res

    Google Scholar 

  2. Howard RA (1963) Semi-Markovian decision-processes. Bull Int Stat Inst

    Google Scholar 

  3. Ross SM (2013) Applied probability models with optimization applications. Courier Corporation

    Google Scholar 

  4. Lippman SA, et al (1971) Maximal average-reward policies for semi-Markov decision processes with arbitrary state and action space. Ann Math Stat

    Google Scholar 

  5. Federgruen A, Hordijk A, Tijms HC (1978) A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J Appl Probab

    Google Scholar 

  6. Yang P, Catthoor F (2003) Pareto-optimization-based run-time task scheduling for embedded systems. In: Proceedings of the 1st IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis

    Google Scholar 

  7. Owen G (1995) Game theory academic press. San Diego

    Google Scholar 

  8. Chatterjee K, Majumdar R, Henzinger TA (2006) Markov decision processes with multiple objectives. In: Annual symposium on theoretical aspects of computer science

    Google Scholar 

  9. Mondal P (2016) On undiscounted semi-Markov decision processes with absorbing states. Math Methods Oper Res

    Google Scholar 

  10. Sinha S, Mondal P (2017) Semi-Markov decision processes with limiting ratio average rewards. J Math Anal Appl

    Google Scholar 

  11. Papadimitriou CH, Yannakakis M (2000) On the approximability of trade-offs and optimal access of web sources. In: Proceedings 41st annual symposium on foundations of computer science

    Google Scholar 

  12. Belenson SM, Kapur KC (2017) An algorithm for solving multicriterion linear programming problems with examples. J Math Anal Appl

    Google Scholar 

  13. Luc DT, Luc DT (2016) Pareto optimality. Multiobjective linear programming: an introduction

    Google Scholar 

  14. Puterman ML (2014) Markov decision processes: discrete stochastic dynamic programming. Wiley

    Google Scholar 

  15. Wessels J, Nunen JAEE van (1975) Discounted semi-Markov decision processes: linear programming and policy iteration. Wiley Online Library

    Google Scholar 

  16. Goldstein AA, Cheney W, et al (1958) A finite algorithm for the solution of consistent linear equations and inequalities and for the Tchebycheff approximation of inconsistent linear equations. Pacific J Math

    Google Scholar 

  17. Garey MR, Johnson DS (1979) Computers and intractability. Freeman San Francisco

    Google Scholar 

  18. Brayton RK, Hachtel GD, Sangiovanni-Vincentelli AL (1981) A survey of optimization techniques for integrated-circuit design. Proc IEEE

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Jadavpur University, 188, Raja S.C. Mallick Rd, Kolkata, 700032, India

    Kushal Guha Bakshi

Authors
  1. Kushal Guha Bakshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Information Technology, Maulana Abul Kalam Azad University of Technology, Haringhata, India

    Debasis Giri

  2. The University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

  3. Department of Mathematics, Indian Institute of Technology Madras, Chennai, India

    Saminathan Ponnusamy

  4. Technical University of Denmark, Kongens Lyngby, Denmark

    Weizhi Meng

  5. Department of Computer Engineering, Ondokuz Mayis University, Atakum, Turkey

    Sedat Akleylek

  6. Department of Information Technology, Indian Institute of Engineering Science and Technology, West Bengal, India

    Santi Prasad Maity

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Guha Bakshi, K. (2022). Semi-Markov Decision Processes with Vector Pay-Offs. In: Giri, D., Raymond Choo, KK., Ponnusamy, S., Meng, W., Akleylek, S., Prasad Maity, S. (eds) Proceedings of the Seventh International Conference on Mathematics and Computing . Advances in Intelligent Systems and Computing, vol 1412. Springer, Singapore. https://doi.org/10.1007/978-981-16-6890-6_76

Download citation

Publish with us

Policies and ethics

Search

Navigation