Skip to main content
SpringerLink
Log in

Experiments with Solving Mountain Car Problem Using State Discretization and Q-Learning

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13757))

Included in the following conference series:

  • 870 Accesses

Abstract

The aim of this paper is to explore the model of the Mountain Car Problem. We provide insight into the physics behind the model. We present some experimental results obtained by numerically simulating the model. We also propose a reinforcement learning approach for deriving an optimal control policy combining model discretization and Q-learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brockman, G., et al.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)

  2. Limpert, E., Stahel, W.A., Abbt, M.: Log-normal Distributions across the sciences: keys and clues: on the charms of statistics, and how mechanical models resembling gambling machines offer a link to a handy way to characterize log-normal distributions, which can provide deeper insight into variability and probability-normal or log-normal: that is the question. BioScience 51(5), 341–352 (2001). https://doi.org/10.1641/0006-3568(2001)051[0341:LNDATS]2.0.CO;2

  3. Moore, A.W.: Efficient memory-based learning for robot control. Ph.D. thesis, Computer Laboratory, University of Cambridge, Cambridge CB3 0FD, United Kingdom (October 1990)

    Google Scholar 

  4. Moore, A.W.: Variable resolution dynamic programming: efficiently learning action maps in multivariate real-valued state-spaces. In: Birnbaum, L.A., Collins, G.C. (eds.) Machine Learning Proceedings 1991, pp. 333–337. Morgan Kaufmann, San Francisco (1991). https://doi.org/10.1016/B978-1-55860-200-7.50069-6

  5. Singh, S.P., Sutton, R.S.: Reinforcement learning with replacing eligibility traces. Mach. Learn. 22(1), 123–158 (1996). https://doi.org/10.1023/A:1018012322525

    Article  MATH  Google Scholar 

  6. Sugiyama, M.: Statistical Reinforcement Learning. Modern Machine Learning Approaches. Chapman and Hall/CRC, Boca Raton (2015)

    Google Scholar 

  7. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2020)

    MATH  Google Scholar 

  8. Tabor, P.: Q learning with just NumPy. Solving the mountain car. Tutorial (2019). https://www.youtube.com/watch?v=rBzOyjywtPw &t=3s. Accessed 7 Jan 2022

Download references

Author information

Authors and Affiliations

  1. University of Craiova, Craiova, Romania

    Amelia Bădică & Costin Bădică

  2. University of Novi Sad, Novi Sad, Serbia

    Mirjana Ivanović

  3. Frankfurt University of Applied Sciences, Frankfurt am Main, Germany

    Doina Logofătu

Authors
  1. Amelia Bădică
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Costin Bădică
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mirjana Ivanović
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Doina Logofătu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Costin Bădică .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Vietnam National University, Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Tien Khoa Tran

  3. Al-Farabi Kazakh National University, Almaty, Kazakhstan

    Ualsher Tukayev

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

  6. University of Newcastle, Newcastle, NSW, Australia

    Edward Szczerbicki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bădică, A., Bădică, C., Ivanović, M., Logofătu, D. (2022). Experiments with Solving Mountain Car Problem Using State Discretization and Q-Learning. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiński, B., Szczerbicki, E. (eds) Intelligent Information and Database Systems. ACIIDS 2022. Lecture Notes in Computer Science(), vol 13757. Springer, Cham. https://doi.org/10.1007/978-3-031-21743-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21743-2_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21742-5

  • Online ISBN: 978-3-031-21743-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation