Abstract
Hierarchical reinforcement learning has been proposed as a solution to the problem of scaling up reinforcement learning. The RL-TOPs Hierarchical Reinforcement Learning System is an implementation of this proposal which structures an agent’s sensors and actions into various levels of representation and control. Disparity between levels of representation means actions can be misused by the planning algorithm in the system. This paper reports on how ILP was used to bridge these representation gaps and shows empirically how this improved the system’s performance. Also discussed are some of the problems encountered when using an ILP system in what is inherently a noisy and incremental domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Proceedings of the 15th International Conference on Machine Learning. Morgan Kaufmann, 1998.
Scott Benson. Learning Action Models for Reactive Autonomous Agents. PhD thesis, Department of Computer Science, Stanford University, 1996.
Thomas G. Dietterich. The maxq method for hierarchical reinforcement learning. In Proceedings of the 15th International Conference on Machine Learning [1].
S. Džeroski, S. Muggleton, and S. Russel. PAC learnability of determinate logic programs. In Proceeding of the Fifth ACM Workshop on Computational Learning Theory, pages 128–135, 1992.
Sašo Džeroski, Luc De Raedt, and Hendrik Blockeel. Relational reinforcement learning. In Proceedings of the 8th International Workshop on Inductive Logic Programming, pages 11–22, 1998.
Yolanda Gil. Learning by experimentation: Incremental refinement of incomplete planning domains. In Proceedings of the 11th International Workshop on Machine Learning, 1994.
Wayne Iba, James Wogulis, and Pat Langley. Trading off simplicity and coverage in incremental concept learning. In Proceedings of the 5th International Conference on Machine Learning, pages 73–79, 1988.
Pat Langley. Elements of Machine Learning. Morgan Kaufmann, 1996.
E. McCreath and A. Sharma. Lime: A system for learning relations. In The 9th International Workshop on Algorithmic Learning Theory. Springer-Verlag, October1998.
N. J. Nilsson. Teleo-reactive programs for agent control. Journal of Artificial Intelligence Research, 1:139–158, 1994.
Ronald Parr and Stuart Russell. Reinforcement learning with hierarchies of machines. In Advances in Neural Information Processing Systems 10: Proceedings of the 1997 Conference, 1998.
Malcolm R. K. Ryan and Mark D. Pendrith. RL-TOPS: An architecture for modularity and re-use in reinforcement learning. In Proceedings of the 15th International Conference on Machine Learning [1].
Malcolm R. K. Ryan and Mark Reid. Learning to fly: An application of hierarchical reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, (to appear).
Wei-Min Shen. Discovery as autonomous learning from the environment. Machine Learning, 12:143–156, 1993.
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.
Richard S. Sutton, Doina Precup, and Satinder Singh. Between mdps and semimdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112:181–211, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Reid, M., Ryan, M. (2000). Using ILP to Improve Planning in Hierarchical Reinforcement Learning. In: Cussens, J., Frisch, A. (eds) Inductive Logic Programming. ILP 2000. Lecture Notes in Computer Science(), vol 1866. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44960-4_11
Download citation
DOI: https://doi.org/10.1007/3-540-44960-4_11
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67795-6
Online ISBN: 978-3-540-44960-7
eBook Packages: Springer Book Archive