Skip to main content
SpringerLink
Log in

A Hybrid POMDP-BDI Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels

  • Conference paper
  • First Online:
Agents and Artificial Intelligence (ICAART 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9494))

Included in the following conference series:

  • 573 Accesses

Abstract

We propose an agent architecture which combines Partially observable Markov decision processes (POMDPs) and the belief-desire-intention (BDI) framework to capitalize on their complimentary strengths. Our architecture introduces the notion of intensity of the desire for a goal’s achievement. We also define an update rule for goals’ desire levels. When to select a new goal to focus on is also defined. To verify that the proposed architecture works, experiments were run with an agent based on the architecture, in a domain where multiple goals must continually be achieved. The results show that (i) while the agent is pursuing goals, it can concurrently perform rewarding actions not directly related to its goals, (ii) the trade-off between goals and preferences can be set effectively and (iii) goals and preferences can be satisfied even while dealing with stochastic actions and perceptions. We believe that the proposed architecture furthers the theory of high-level autonomous agent reasoning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \( Pref (\cdot )\) is designed such that the agent collects a maximum number of items (ignoring goals). The agent collects more when it is encouraged to sense where items are, hence \( sensUtil \) is 1 if the agent tries to \( see \).

  2. 2.

    Essentially, the goals in G are stacked in descending order of the value of \(V^*_ HPB (B,g,h^-)\), where \(h^- < h\) and B is the current belief-state. The goal on top of the stack becomes the intention.

References

  1. Bratman, M.: Intention, Plans, and Practical Reason. Harvard University Press, Massachusetts (1987)

    Google Scholar 

  2. Rao, A., Georgeff, M.: BDI agents: From theory to practice. In: Proceedings of the ICMAS 1995, pp. 312–319. AAAI Press (1995)

    Google Scholar 

  3. Monahan, G.: A survey of partially observable Markov decision processes: theory, models, and algorithms. Manage. Sci. 28, 1–16 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  4. Lovejoy, W.: A survey of algorithmic methods for partially observed Markov decision processes. Ann. Oper. Res. 28, 47–66 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  5. Koenig, S.: Agent-centered search. Artif. Intell. Mag. 22, 109–131 (2001)

    Google Scholar 

  6. Ross, S., Pineau, J., Paquet, S., Chaib-draa, B.: Online planning algorithms for POMDPs. J. Artif. Intell. Res. (JAIR) 32, 663–704 (2008)

    MathSciNet  MATH  Google Scholar 

  7. Schut, M., Wooldridge, M., Parsons, S.: The theory and practice of intention reconsideration. Exp. Theor. Artif. Intell. 16, 261–293 (2004)

    Article  Google Scholar 

  8. Wooldridge, M.: Intelligent agents. In: Weiss, G. (ed.) Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, Massachusetts (1999)

    Google Scholar 

  9. Wooldridge, M.: An Introduction to Multiagent Systems. Wiley, Chichester (2002)

    Google Scholar 

  10. Wooldridge, M.: Reasoning About Rational Agents. MIT Press, Massachusetts (2000)

    MATH  Google Scholar 

  11. Schut, M., Wooldridge, M.: Principles of intention reconsideration. In: Agents 2001: Proceedings of the 5th International Conference on Autonomous Agents, pp. 340–347. ACM Press, New York (2001)

    Google Scholar 

  12. Pollack, M., Ringuette, M.: Introducing the tileworld: experimentally evaluating agent architectures. In: Proceedings of the AAAI 1990, pp. 183–189. AAAI Press (1990)

    Google Scholar 

  13. Kinny, D., Georgeff, M.: Commitment and effectiveness of situated agents. In: Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), pp. 82–88 (1991)

    Google Scholar 

  14. Kinny, D., Georgeff, M.: Experiments in optimal sensing for situated agents. In: Proceedings of the 2nd Pacific Rim Internatioanl Conference on Artificial Intelligence (PRICAI 1992) (1992)

    Google Scholar 

  15. Schut, M., Wooldridge, M.: Intention reconsideration in complex environments. In: Proceedings of the 4th International Conference on Autonomous Agents (AGENTS 2000). ACM, New York (2000)

    Google Scholar 

  16. Schut, M., Wooldridge, M.: The control of reasoning in resource-bounded agents. Knowl. Eng. Rev. 16, 215–240 (2001)

    Article  Google Scholar 

  17. Kaelbling, L., Littman, M., Cassandra, A.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  18. Walczak, A., Braubach, L., Pokahr, A., Lamersdorf, W.: Augmenting BDI agents with deliberative planning techniques. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) PROMAS 2006. LNCS (LNAI), vol. 4411, pp. 113–127. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  19. Meneguzzi, F., Zorzo, A., Móra, M., Luck, M.: Incorporating planning into BDI systems. Scalable Comput. Pract. Experience 8, 15–28 (2007)

    Google Scholar 

  20. Nair, R., Tambe, M.: Hybrid bdi-pomdp framework for multiagent teaming. J. Artif. Intell. Res. (JAIR) 23, 367–420 (2005)

    MATH  Google Scholar 

  21. Lim, M.Y., Dias, J., Aylett, R.S., Paiva, A.C.R.: Improving adaptiveness in autonomous characters. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 348–355. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  22. Pereira, D., Gonçalves, L., Dimuro, G., Costa, A.: Constructing bdi plans from optimal pomdp policies, with an application to agentspeak programming. In: Henning, G., Galli, M., Goneet, S. (eds.) XXXIV Conferência Latinoamericano de Informática, Santa Fe. Anales CLEI 2008, pp. 240–249 (2008)

    Google Scholar 

  23. Simari, G., Parsons, S.: On the relationship between mdps and the bdi architecture. In: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2006, pp. 1041–1048. ACM, New York (2006)

    Google Scholar 

  24. Simari, G., Parsons, S.: Markov Decision Processes and the Belief-Desire-Intention Model. Springer Briefs in Computer Science. Springer, Heidelberg (2011)

    Book  MATH  Google Scholar 

  25. Rens, G., Ferrein, A., Van der Poel, E.: A BDI agent architecture for a POMDP planner. In: Lakemeyer, G., Morgenstern, L., Williams, M.A. (eds.) Proceedings of the 9th International Symposium on Logical Formalizations of Commonsense Reasoning (Commonsense 2009), University of Technology, pp. 109–114. UTSe Press, Sydney (2009)

    Google Scholar 

  26. Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S.: Decision-theoretic, high-level agent programming in the situation calculus. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI 2000) and of the Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI 2000), pp. 355–362. AAAI Press, Menlo Park (2000)

    Google Scholar 

  27. Chen, Y., Hong, J., Liu, W., Godo, L., Sierra, C., Loughlin, M.: Incorporating PGMs into a BDI architecture. In: Boella, G., Elkind, E., Savarimuthu, B.T.R., Dignum, F., Purvis, M.K. (eds.) PRIMA 2013. LNCS, vol. 8291, pp. 54–69. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  28. Antos, D., Pfeffer, A.: Using emotions to enhance decision-making. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), pp. 24–30. AAAI Press, Menlo Park (2011)

    Google Scholar 

  29. Murphy, R.: Introduction to AI Robotics. MIT Press, Massachusetts (2000)

    Google Scholar 

  30. Roy, N., Gordon, G., Thrun, S.: Finding approximate POMDP solutions through belief compressions. J. Artif. Intell. Res. (JAIR) 23, 1–40 (2005)

    Article  MATH  Google Scholar 

  31. Paquet, S., Tobin, L., Chaib-draa, B.: Real-time decision making for large POMDPs. In: Kégl, B., Lee, H.-H. (eds.) Canadian AI 2005. LNCS (LNAI), vol. 3501, pp. 450–455. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  32. Li, X., Cheung, W., Liu, J.: Towards solving large-scale POMDP problems via spatio-temporal belief state clustering. In: Proceedings of IJCAI-05 Workshop on Reasoning with Uncertainty in Robotics (RUR 2005) (2005)

    Google Scholar 

  33. Shani, G., Brafman, R., Shimony, S.: Forward search value iteration for POMDPs. In: de Mantaras, R.L. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2619–2624. AAAI Press, Menlo Park (2007)

    Google Scholar 

  34. Cai, C., Liao, X., Carin, L.: Learning to explore and exploit in pomdps. In: NIPS, pp. 198–206 (2009)

    Google Scholar 

  35. Shani, G., Pineau, J., Kaplow, R.: A survey of point-based pomdp solvers. Auton. Agent. Multi-Agent Syst. 27, 1–51 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal, Durban, South Africa

    Gavin Rens

  2. Centre for Artificial Intelligence Research, CSIR Meraka, Pretoria, South Africa

    Gavin Rens & Thomas Meyer

  3. Department of Computer Science, University of Cape Town, Cape Town, South Africa

    Thomas Meyer

Authors
  1. Gavin Rens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Meyer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gavin Rens .

Editor information

Editors and Affiliations

  1. LERIA - UFR Sciences , Angers, France

    Béatrice Duval

  2. Leiden University , Leiden, Zuid-Holland, The Netherlands

    Jaap van den Herik

  3. LERIA - UFR Sciences , Angers, France

    Stephane Loiseau

  4. Polytechnic Institute of Setúbal , Setúbal, Portugal

    Joaquim Filipe

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Rens, G., Meyer, T. (2015). A Hybrid POMDP-BDI Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels. In: Duval, B., van den Herik, J., Loiseau, S., Filipe, J. (eds) Agents and Artificial Intelligence. ICAART 2015. Lecture Notes in Computer Science(), vol 9494. Springer, Cham. https://doi.org/10.1007/978-3-319-27947-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27947-3_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27946-6

  • Online ISBN: 978-3-319-27947-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation