Skip to main content
SpringerLink
Log in

Parallelizing Parallel Rollout Algorithm for Solving Markov Decision Processes

  • Conference paper
  • First Online:
OpenMP Shared Memory Parallel Programming (WOMPAT 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2716))

Included in the following conference series:

Abstract

Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, 1994.

    MATH  Google Scholar 

  2. D. P. Bertsekas and J. N. Tsitsiklis. Neuro dynamic programming. Athena Scientific, 1996.

    Google Scholar 

  3. R. Sutton and A. Barto. Reinforcement Learning. MIT Press, 2000.

    Google Scholar 

  4. A. Printista, M. Errecalde, and C. Montoya. A parallel implementation of Qlearning based on communication with cache. Journal of Computer Science and Technology, 1(6), 2002.

    Google Scholar 

  5. H. S. Chang, R. Givan, and E. K. P. Chong. Parallel rollout for on-line solution of partially observable markov decision processes. Discrete Event Dynamic Systems (Revised), 2002.

    Google Scholar 

  6. M. Littman, T. Dean, and L. Kaelbling. On the complexity of solving markov decision problems. In Proc. 11th Annual Conf. on Uncertainty in Artificial Intelligence, pages 394–402, 1995.

    Google Scholar 

  7. A. M. Law and W. D. Kelton. Simulation Modeling and Analysis, 3rd Ed. McGraw-Hill, New York, 2000.

    Google Scholar 

  8. D. P. Bertsekas. Differential training of rollout policies. In Proc. 35th Allerton Conf. on Comm., Control, and Computing, 1997.

    Google Scholar 

  9. R. Givan, E. K. P. Chong, and H. S. Chang. Scheduling multiclass packet streams to minimize weighted loss. Queueing Systems, 41:241–270, 2002.

    Article  MATH  MathSciNet  Google Scholar 

  10. W. Fischer and K. Meier-Hellstern. The markov-modulated poisson process (mmpp) cookbook. Performance Evaluation, 18:149–171, 1992.

    Article  MathSciNet  Google Scholar 

  11. Wolfgang E. Nagel, Alfred Arnold, Michael Weber, and Hans-Christian Hoppe. VAMPIR: Visualization and analysis of MPI resources. Supercomputer, (1):69–80, January 1996.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Advanced Computer Systems Laboratory, Department of Electronics and Computer Engineering, Korea University, Seoul, Korea

    Seon Wook Kim

  2. Department of Computer Science and Engineering, Sogang University, Seoul, Korea

    Hyeong Soo Chang

Authors
  1. Seon Wook Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyeong Soo Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, Ontario, M5S 3G4, Canada

    Michael J. Voss

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kim, S.W., Chang, H.S. (2003). Parallelizing Parallel Rollout Algorithm for Solving Markov Decision Processes. In: Voss, M.J. (eds) OpenMP Shared Memory Parallel Programming. WOMPAT 2003. Lecture Notes in Computer Science, vol 2716. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45009-2_10

Download citation

  • DOI: https://doi.org/10.1007/3-540-45009-2_10

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40435-4

  • Online ISBN: 978-3-540-45009-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation