Abstract
This paper presents a new method by which a sequential decision agent can automatically discover subgoals online. The agent discovers subgoals using potential field. The method uses a reward function to generate a potential field, and then abstracts some features from the potential field as candidates of subgoals. Based on the candidates, the agent can determine its behaviors online through some heuristics in unknown environment. The best-known and most often-cited problem with the potential field method is local minima. But our method does not have this limitation because the local minima are used to form subgoals. The disadvantage of the local minima in the previous approaches of potential field turns out to be an advantage in our method. We illustrate the method using a simple gridworld task.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Amarel, S.: On representations of problems of reasoning about actions. In: Michie, D. (ed.) Machine Intelligence, 3rd edn., pp. 131–171 (1968)
Anzai, Y., Simon, H.A.: The theory of learning by doing. Psychological Review 86, 124–140 (1979)
Iba, G.A.: A heuristic approach to the discovery of macro-operators. Machine Learning 3, 285–317 (1989)
Digney, B.: Emergent hierarchical control structures: Learning reactive hierarchical relationships in reinforcement environments. From Animals to Animats 4 (1996)
Digney, B.: Learning hierarchical control structure for multiple tasks and changing environments. From Animals to Animats 5 (1998)
Drummond, C.: Composing functions to speed up reinforcement learning in a changing world. In: European Conference on Machine Learning, pp. 370–381 (1998)
McGovern, A., Barto, A.G.: Automatic discovery of subgoals in reinforcement learning using diverse density. In: Proceedings of the 18th International Conference on Machine Learning, pp. 361–368 (2001)
Arkin, R.C.: Intelligent Robots and Autonomous Agents. The MIT Press, Cambridge (1998)
Koren, Y., Borenstein, J.: Potential field methods and their inherent limitations for mobile robot navigation. In: Proceedings of the IEEE Conference on Robotics and Automation, pp. 1398–1404 (1991)
Hwang, Y.K., Ahuja, N.: A potential field approach to path planning. IEEE Transaction on Robots and Automation 8, 23–32 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, H., Yin, C., Xie, L. (2005). Automatic Discovery of Subgoals for Sequential Decision Problems Using Potential Fields. In: Wang, L., Chen, K., Ong, Y.S. (eds) Advances in Natural Computation. ICNC 2005. Lecture Notes in Computer Science, vol 3612. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11539902_46
Download citation
DOI: https://doi.org/10.1007/11539902_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28320-1
Online ISBN: 978-3-540-31863-7
eBook Packages: Computer ScienceComputer Science (R0)