Skip to main content
Springer Nature Link
Log in

Comparing real-time and incremental heuristic search for real-time situated agents

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Real-time situated agents, such as characters in real-time computer games, often do not know the terrain in advance but automatically observe it within a certain range around themselves. They have to interleave searches with action executions to make the searches tractable when moving autonomously to user-specified coordinates. The searches face real-time requirements since it is important that the agents be responsive to the commands of the users and move smoothly. In this article, we compare two classes of fast heuristic search methods for these navigation tasks that speed up A* searches in different ways, namely real-time heuristic search and incremental heuristic search, to understand their advantages and disadvantages and make recommendations about when each one should be used. We first develop a competitive real-time heuristic search method. LSS-LRTA* is a version of Learning Real-Time A* that uses A* to determine its local search spaces and learns quickly. We analyze the properties of LSS-LRTA* and then compare it experimentally against the state-of-the-art incremental heuristic search method D* Lite on our navigation tasks, for which D* Lite was specifically developed, resulting in the first comparison of real-time and incremental heuristic search in the literature. We characterize when to choose each one of the two heuristic search methods, depending on the search objective and the kind of terrain. Our experimental results show that LSS-LRTA* can outperform D* Lite under the right conditions, namely when there is time pressure or the user-supplied h-values are generally not misleading.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Barto A., Bradtke S. and Singh S. (1995). Learning to act using real-time dynamic programming. Artificial Intelligence 73(1): 81–138

    Article  Google Scholar 

  2. Bjornsson, M., Enzenberger, M., Holte, R., Schaeffer, J., & Yap, P. (2003). Comparison of different abstractions for pathfinding on maps. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 1511–1512).

  3. Bonet, B., & Geffner, H. (2000). Planning with incomplete information as heuristic search in belief space. In Proceedings of the International Conference on Artificial Intelligence Planning and Scheduling (pp. 52–61).

  4. Bonet, B., Loerincs, G., & Geffner, H. (1997). A robust and fast action selection mechanism. In Proceedings of the National Conference on Artificial Intelligence (pp. 714–719).

  5. Bulitko, V. (2003). Lookahead pathologies and meta-level control in real-time heuristic search. In Proceedings of the Euromicro Conference on Real-Time Systems (pp. 13–16).

  6. Bulitko, V., Bjornsson, Y., Luvstrek, M., Schaeffer, J., & Sigmundarson, S. (2007). Dynamic control in path-planning with real-time heuristic search. In Proceedings of the International Conference on Automated Planning and Scheduling (pp. 49–56).

  7. Bulitko V. and Lee G. (2006). Learning in real-time search: A unifying framework. Journal of Artificial Intelligence Research 25: 119–157

    Google Scholar 

  8. Dijkstra E. (1959). A note on two problems in connexion with graphs. Numerische Mathematik 1: 269–271

    Article  MATH  MathSciNet  Google Scholar 

  9. Edelkamp, S. (1998). Updating shortest paths. In Proceedings of the European Conference on Artificial Intelligence (pp. 655–659).

  10. Furcy, D., & Koenig, S. (2000). Speeding up the convergence of real-time search. In Proceedings of the National Conference on Artificial Intelligence (pp. 891–897).

  11. Goldenberg, M., Kovarksy, A., Wu, X., & Schaeffer, J. (2003). Multiple agents moving target search. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 1538–1538).

  12. Hart, P., Nilsson, N., & Raphael, N. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, SSC4(2), 100–107.

    Google Scholar 

  13. Hart P., Nilsson N. and Raphael B. (1972). Correction to ‘a formal basis for the heuristic determination of minimum cost paths’. SIGART Newsletter 37: 28–29

    Google Scholar 

  14. Hebert, M., McLachlan, R., & Chang, P. (1999). Experiments with driving modes for urban robots. In Proceedings of the SPIE Mobile Robots.

  15. Holte R., Mkadmi T., Zimmer R. and MacDonald A. (1996). Speeding up problem solving by abstraction: A graph oriented approach. Artificial Intelligence 85(1–2): 321–361

    Article  Google Scholar 

  16. Ishida, T. (1992). Moving target search with intelligence. In Proceedings of the National Conference on Artificial Intelligence (pp. 525–532).

  17. Ishida, T. (1997). Real-Time search for learning autonomous agents. Kluwer Academic Publishers.

  18. Koenig S. (2001). Agent-centered search. Artificial Intelligence Magazine 22(4): 109–131

    MathSciNet  Google Scholar 

  19. Koenig S. (2001). Minimax real-time heuristic search. Artificial Intelligence 129: 165–197

    Article  MATH  MathSciNet  Google Scholar 

  20. Koenig, S. (2004). A comparison of fast search methods for real-time situated agents. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (pp. 864–871).

  21. Koenig, S., & Likhachev, M. (2002). D* Lite. In Proceedings of the National Conference on Artificial Intelligence (pp. 476–483).

  22. Koenig S. and Likhachev M. (2005). Fast replanning for navigation in unknown terrain. Transactions on Robotics 21(3): 354–363

    Article  Google Scholar 

  23. Koenig, S., & Likhachev, M. (2006). A new principle for incremental heuristic search: Theoretical results. In Proceedings of the International Conference on Autonomous Planning and Scheduling (pp. 402–405).

  24. Koenig S., Likhachev M. and Furcy D. (2004). Lifelong Planning A*. Artificial Intelligence Journal 155(1–2): 93–146

    Article  MATH  MathSciNet  Google Scholar 

  25. Koenig S., Likhachev M., Liu Y. and Furcy D. (2004). Incremental heuristic search in Artificial Intelligence. Artificial Intelligence Magazine 25(2): 99–112

    Google Scholar 

  26. Koenig, S., & Simmons, R. G. (1996). Easy and hard testbeds for real-time search algorithms. In Proceedings of the National Conference on Artificial Intelligence (pp. 279–285).

  27. Koenig, S., & Szymanski, B. (1999). Value-update rules for real-time search. In Proceedings of the National Conference on Artificial Intelligence (pp. 718–724).

  28. Koenig S., Tovey C. and Smirnov Y. (2003). Performance bounds for planning in unknown terrain. Artificial Intelligence 147: 253–279

    Article  MATH  MathSciNet  Google Scholar 

  29. Korf R. (1990). Real-time heuristic search. Artificial Intelligence 42(2–3): 189–211

    Article  MATH  Google Scholar 

  30. Korf R. (1993). Linear-space best-first search. Artificial Intelligence 62(1): 41–78

    Article  MATH  MathSciNet  Google Scholar 

  31. Mudgal, A., Tovey, C., & Koenig, S. (2004). Analysis of greedy robot-navigation methods. In Proceedings of the Conference on Artificial Intelligence and Mathematics.

  32. Pearl, J. (1985). Heuristics: Intelligent search strategies for computer problem solving. Addison-Wesley.

  33. Pemberton, J., & Korf, R. (1992). Making locally optimal decisions on graphs with cycles. Technical Report 920004. Los Angeles, CA: Computer Science Department, University of California at Los Angeles.

  34. Russell, S., & Wefald, E. (1991). Do the right thing—Studies in limited rationality. MIT Press.

  35. Shue, L., Li, S., & Zamani, R. (2001). An intelligent heuristic algorithm for project scheduling problems. In Proceedings of the Annual Meeting of the Decision Sciences Institute.

  36. Shue, L., & Zamani, R. (1993). An admissible heuristic search algorithm. In Proceedings of the International Symposium on Methodologies for Intelligent Systems (pp. 69–75).

  37. Stentz, A. (1995). The focussed D* algorithm for real-time replanning. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 1652–1659).

  38. Sun, X., & Koenig, S. (2007). The Fringe-Saving A* search algorithm—A feasibility study. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 2391–2397).

  39. Svennebring J. and Koenig S. (2004). Building terrain-covering ant robots. Autonomous Robots 16(3): 313–332

    Article  Google Scholar 

  40. Thayer, S., Digney, B., Diaz, M., Stentz, A., Nabbe, B., & Hebert, M. (2000). Distributed robotic mapping of extreme environments. In Proceedings of the SPIE: Mobile Robots XV and Telemanipulator and Telepresence Technologies VII (Vol. 4195, pp. 84–95).

  41. Trovato K. (1990). Differential A*: An adaptive search method illustrated with robot path planning for moving obstacles and goals and an uncertain environment. Journal of Pattern Recognition and Artificial Intelligence 4(2): 245–268

    Article  Google Scholar 

  42. Wagner I., Lindenbaum M. and Bruckstein A. (1999). Distributed covering by ant-robots using evaporating traces. IEEE Transactions on Robotics and Automation 15(5): 918–933

    Article  Google Scholar 

  43. Yanovski V., Wagner I. and Bruckstein A. (2003). A distributed ant algorithm for efficiently patrolling a network. Algorithmica 37: 165–186

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Southern California, Los Angeles, CA, 90089-0781, USA

    Sven Koenig & Xiaoxun Sun

Authors
  1. Sven Koenig
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoxun Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sven Koenig.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Koenig, S., Sun, X. Comparing real-time and incremental heuristic search for real-time situated agents. Auton Agent Multi-Agent Syst 18, 313–341 (2009). https://doi.org/10.1007/s10458-008-9061-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-008-9061-x

Keywords