Skip to main content

Incremental Learning of Planning Operators in Stochastic Domains

  • Conference paper
SOFSEM 2007: Theory and Practice of Computer Science (SOFSEM 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4362))

Abstract

In this work we assume that there is an agent in an unknown environment (domain). This agent has some predefined actions and it can perceive its current state in the environment completely. The mission of this agent is to fulfill the tasks (goals) that are often assigned to it as fast as it can. Acting has lots of cost, and usually planning and simulating the environment can reduce this cost. In this paper we address a new approach for incremental induction of probabilistic planning operators, from this environment while the agent tries to reach to its current goals. It should be noted that there have been some works related to incremental induction of deterministic planning operators and batch learning of probabilistic planning operators, but the problem of incremental induction of probabilistic planning operators has not been studied yet. We also address some trade offs such as exploration (for better learning of stochastic operators, acting) and exploitation (for fast discovery of goals, planning), and we explain that a good decision in these trade offs is dependant on the stability and accuracy of the learned planning operators.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Kaelbling, L., Littman, H., Moore, A.: Reinforcement Learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)

    Google Scholar 

  2. Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD Thesis, King’s College, Cambridge, UK (1989)

    Google Scholar 

  3. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (1998)

    Google Scholar 

  4. Boutilier, C., Dean, T., Hanks, S.: Decision-Theoretic Planning: Structural Assumptions and Computational Leverage. Journal of Artificial Intelligence Research 11, 1–94 (1999)

    MATH  MathSciNet  Google Scholar 

  5. Gil, Y.: Learning by Experimentation: Incremental Refinement of Incomplete Planning Domains. In: Eleventh International Conference on Machine Learning (1994)

    Google Scholar 

  6. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  7. Wang, X.: Learning by Observation and Practice: An Incremental Approach for Planning Operator Acquisition. In: Twelfth International Conference on Machine Learning (1995)

    Google Scholar 

  8. Veloso, M., Carbonell, J., Pérez, A., Borrajo, D., Fink, E., Blythe, J.: Integrating Planning and Learning: The PRODIGY Architecture. Journal of Experimental and Theoretical Artificial Intelligence 7, 81–120 (1995)

    Article  MATH  Google Scholar 

  9. Oates, T., Cohen, P.R.: Learning Planning Operators with Conditional and Probabilistic Effects. In: AAAI Symposium on Planning with Incomplete Information for Robot Problems (1996)

    Google Scholar 

  10. Pasula, H., Zettlemoyer, L.S., Kaelbling, L.P.: Learning Probabilistic Relational Planning Rules. In: Fourteenth International Conference on Automated Planning and Scheduling (2004)

    Google Scholar 

  11. Zettlemoyer, L.S., Pasula, H., Kaelbling, L.P.: Learning Planning Rules in Noisy Stochastic Worlds. In: Proceedings of the Twentieth National Conference on Artificial Intelligence, AAAI-05 (2005)

    Google Scholar 

  12. ICAPS-06, the 5th International Planning Competition IPC-5 (2006), http://www.ldc.usb.ve/~bonet/ipc5/

  13. Littman, M.L., Younes, H.L.S.: IPC 2004 Probabilistic Planning Track: FAQ 0.1. In: Proceedings of the ICAPS-03 Workshop on the Competition: Impact, Organization, Evaluation, Benchmarks, pp. 7–12 (2003)

    Google Scholar 

  14. Muggleton, S., Raedt, L.D.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19, 629–679 (1994)

    Article  MathSciNet  Google Scholar 

  15. Ghallab, M., Nau, D., Traverso, P.: Automated Planning: Theory and Practice. Morgan Kaufmann, San Francisco (2004)

    MATH  Google Scholar 

  16. Sutton, R.S.: Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming. In: Proceedings of the Seventh International Conference on Machine Learning, pp. 216–224. Morgan Kaufmann, San Mateo (1990)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Jan van Leeuwen Giuseppe F. Italiano Wiebe van der Hoek Christoph Meinel Harald Sack František Plášil

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Safaei, J., Ghassem-Sani, G. (2007). Incremental Learning of Planning Operators in Stochastic Domains. In: van Leeuwen, J., Italiano, G.F., van der Hoek, W., Meinel, C., Sack, H., Plášil, F. (eds) SOFSEM 2007: Theory and Practice of Computer Science. SOFSEM 2007. Lecture Notes in Computer Science, vol 4362. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69507-3_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-69507-3_56

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-69506-6

  • Online ISBN: 978-3-540-69507-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics