Abstract
In this paper we present a new method for improving reinforcement learning training times under the following two assumptions: (1) we know the conditions under which the environment gives reward; and (2) we can control the initial state of the environment at the beginning of a training episode. Our method, called intra-task curriculum learning, presents the different episode starting states to an agent in order of increasing distance to immediate reward.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Eligibility traces are recorded for each value in value function based reinforcement learning. We use the term “state” here, but in practice the values can also be stored for state-action pairs (e.g. in Q-learning).
- 2.
For the purpose of this paper, we define the distance form state \(s_a\) to state \(s_b\) as the minimum number of transitions required to get from \(s_a\) to \(s_b\). This definition is sufficient because the environments we use have deterministic transitions, but a different definition would be required for stochastic environments.
- 3.
We have a \(5 \times 5\) grid with 7 walls; leaving 18 free spaces.
- 4.
We number the states in the 2D environment from left to right, bottom to top.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Pathak, D., et al.: Curiosity-driven exploration by self-supervised prediction. In: ICML (2017)
Chentanez, N., Barto, A.G., Singh, S.P.: Intrinsically motivated reinforcement learning. In: NIPS, pp. 1281–1288 (2005)
Dosovitskiy, A., Koltun, V.: Learning to act by predicting the future. In: ICLR (2017)
Bengio, Y., et al.: Curriculum learning. In: ICML, pp. 41–48 (2009)
van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: AAAI (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
du Preez-Wilkinson, N., Gallagher, M., Hu, X. (2018). Intra-task Curriculum Learning for Faster Reinforcement Learning in Video Games. In: Mitrovic, T., Xue, B., Li, X. (eds) AI 2018: Advances in Artificial Intelligence. AI 2018. Lecture Notes in Computer Science(), vol 11320. Springer, Cham. https://doi.org/10.1007/978-3-030-03991-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-03991-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03990-5
Online ISBN: 978-3-030-03991-2
eBook Packages: Computer ScienceComputer Science (R0)