Skip to main content

Automatic Discovery of Subgoals for Sequential Decision Problems Using Potential Fields

  • Conference paper
Advances in Natural Computation (ICNC 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3612))

Included in the following conference series:

Abstract

This paper presents a new method by which a sequential decision agent can automatically discover subgoals online. The agent discovers subgoals using potential field. The method uses a reward function to generate a potential field, and then abstracts some features from the potential field as candidates of subgoals. Based on the candidates, the agent can determine its behaviors online through some heuristics in unknown environment. The best-known and most often-cited problem with the potential field method is local minima. But our method does not have this limitation because the local minima are used to form subgoals. The disadvantage of the local minima in the previous approaches of potential field turns out to be an advantage in our method. We illustrate the method using a simple gridworld task.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Amarel, S.: On representations of problems of reasoning about actions. In: Michie, D. (ed.) Machine Intelligence, 3rd edn., pp. 131–171 (1968)

    Google Scholar 

  2. Anzai, Y., Simon, H.A.: The theory of learning by doing. Psychological Review 86, 124–140 (1979)

    Article  Google Scholar 

  3. Iba, G.A.: A heuristic approach to the discovery of macro-operators. Machine Learning 3, 285–317 (1989)

    Google Scholar 

  4. Digney, B.: Emergent hierarchical control structures: Learning reactive hierarchical relationships in reinforcement environments. From Animals to Animats 4 (1996)

    Google Scholar 

  5. Digney, B.: Learning hierarchical control structure for multiple tasks and changing environments. From Animals to Animats 5 (1998)

    Google Scholar 

  6. Drummond, C.: Composing functions to speed up reinforcement learning in a changing world. In: European Conference on Machine Learning, pp. 370–381 (1998)

    Google Scholar 

  7. McGovern, A., Barto, A.G.: Automatic discovery of subgoals in reinforcement learning using diverse density. In: Proceedings of the 18th International Conference on Machine Learning, pp. 361–368 (2001)

    Google Scholar 

  8. Arkin, R.C.: Intelligent Robots and Autonomous Agents. The MIT Press, Cambridge (1998)

    Google Scholar 

  9. Koren, Y., Borenstein, J.: Potential field methods and their inherent limitations for mobile robot navigation. In: Proceedings of the IEEE Conference on Robotics and Automation, pp. 1398–1404 (1991)

    Google Scholar 

  10. Hwang, Y.K., Ahuja, N.: A potential field approach to path planning. IEEE Transaction on Robots and Automation 8, 23–32 (1992)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Chen, H., Yin, C., Xie, L. (2005). Automatic Discovery of Subgoals for Sequential Decision Problems Using Potential Fields. In: Wang, L., Chen, K., Ong, Y.S. (eds) Advances in Natural Computation. ICNC 2005. Lecture Notes in Computer Science, vol 3612. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11539902_46

Download citation

  • DOI: https://doi.org/10.1007/11539902_46

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-28320-1

  • Online ISBN: 978-3-540-31863-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics