Skip to main content
Springer Nature Link
Log in

Learning from History for Behavior-Based Mobile Robots in Non-Stationary Conditions

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Learning in the mobile robot domain is a very challenging task, especially in nonstationary conditions. The behavior-based approach has proven to be useful in making mobile robots work in real-world situations. Since the behaviors are responsible for managing the interactions between the robots and its environment, observing their use can be exploited to model these interactions. In our approach, the robot is initially given a set of “behavior-producing” modules to choose from, and the algorithm provides a memory-based approach to dynamically adapt the selection of these behaviors according to the history of their use. The approach is validated using a vision- and sonar-based Pioneer I robot in nonstationary conditions, in the context of a multirobot foraging task. Results show the effectiveness of the approach in taking advantage of any regularities experienced in the world, leading to fast and adaptable specialization for the learning robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Agre, P.E. 1988. The dynamic structure of everyday life. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Albus, J.S. 1991. Outline for a theory of intelligence. IEEE Trans. on Systems, Man, and Cybernetics, 21(3):473-509.

    Google Scholar 

  • Asada, M., Uchibe, E., Noda, S., Tawaratsumida, S., and Hosoda, K. 1994. Coordination of multiple behaviors acquired by a vision-based reinforcement learning. In Proc. IEEE/RSJ/GI Int'l Conf. on Intelligent Robots and Systems, Munich, Germany.

  • Brooks, R.A. 1986. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, RA-2(1):14- 23.

    Google Scholar 

  • Brooks, R.A. 1991. Intelligence without representation. Artificial Intelligence, 47:139-159.

    Google Scholar 

  • Brooks, R.A. 1996. MARS: Multiple agency reactivity system. Technical Report IS Robotics, Cambridge, MA.

    Google Scholar 

  • del R. Millán, J. 1996. Rapid, safe, and incremental learning of navigation strategies, IEEE Trans. on Systems, Man, and Cybernetics—Part B: Cybernetics, 26(3):408-420.

    Google Scholar 

  • Dorigo, M. and Colombetti, M. 1994. Robot shaping: Developing autonomous agents through learning. Artificial Intelligence, 71(4):321-370.

    Google Scholar 

  • Floreano, D. and Mondada, F. 1996. Evolution of homing navigation in a real mobile robot. IEEE Trans. on Systems, Man, and Cybernetics—Part B: Cybernetics, 26(3):396-407.

    Google Scholar 

  • Fontán, M.S. and Matarić, M.J. 1996. A study of territoriality: the role of critical mass in adaptive task division. In From Animals to Animats: Proc. 4th Int'l Conf. on Simulation of Adaptive Behavior, P. Maes, M.J. Matarić, J.-A. Meyer, J. Pollack, and S. Wilson (Eds.), Cape Cod, The MIT Press.

    Google Scholar 

  • Goldberg, D. and Matarić, M.J. 1997. Interference as a tool for designing and evaluating multirobot controllers. In Proc. National Conf. on Artificial Intelligence (AAAI), Providence, RI, pp. 637- 642.

  • Kaelbling, L.P., Littman, M.L., and Moore, A.W. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285.

    Google Scholar 

  • Maes, P. 1989. The dynamics of action selection. In Proc. Int'l Joint Conf. on Artificial Intelligence (IJCAI), Detroit, MI, pp. 991-997.

  • Maes, P. and Brooks, R.A. 1990. Learning to coordinate behaviors. In Proc. Nat'l Conf. on Artificial Intelligence (AAAI), vol. 2, pp. 796- 802.

    Google Scholar 

  • Mahadevan, S. and Connell, J. 1992. Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55:311-365.

    Google Scholar 

  • Mahadevan, S. and Kaelbling, L.P. 1996. The NSF workshop on reinforcement learning: Summary and observations. AIMagazine.

  • Matarić, M.J. 1992a. Integration of representation into goal-driven behavior-based robots. IEEE Transactions on Robotics and Automation, 8(3):46-54.

    Google Scholar 

  • Matarić, M.J. 1992b. Behavior-based systems: Key properties and implications. In Proc. IEEE Int'l Conf. on Robotics and Automation, Workshop on Architectures for Intelligent Control Systems, Nice, France, pp. 46-54.

  • Matarić, M.J. 1994a. Interaction and intelligent behavior. MIT AI Lab AI-TR 1495, Massachusetts Institute of Technology, Cambridge, MA.

    Google Scholar 

  • Matarić, M.J. 1994b. Reward functions for accelerated learning. In Proc. 11th Int'l Conf. on Machine Learning, W.W. Cohen and H. Hirsh (Eds.), New Brunswick, NJ, Morgan Kauffman Publishers, pp. 181-189.

    Google Scholar 

  • Matarić, M.J. 1997. Reinforcement learning in the multi-robot domain. Autonomous Robots, 4(1).

  • McCallum, A.K. 1996a. Hidden state and reinforcement learning with instance-based state identification. IEEE Trans. on Systems, Man, and Cybernetics—Part B: Cybernetics, 26(3):464- 473.

    Google Scholar 

  • McCallum, A.K. 1996b. Learning to use selective attention and short-term memory in sequential tasks. In From Animals to Animats: Proc. 4th Int'l Conf. on Simulation of Adaptive Behavior, P. Maes, M.J. Matarić, J.-A. Meyer, J. Pollack, and S.W. Wilson (Eds.), Cape Cod, The MIT Press, pp. 315-324.

    Google Scholar 

  • McCallum, A.K. 1996c. Reinforcement learning with selective perception and hidden state. Ph.D. Thesis, Department of Computer Science, University of Rochester, USA.

    Google Scholar 

  • Michaud, F. 1996. Nouvelle architecture unifiée de contrôle intelligent par sélection intentionnelle de comportements. Ph.D. Thesis, Department of Electrical and Computer Engineering, Université de Sherbrooke, Québec, Canada.

    Google Scholar 

  • Michaud, F. 1997. Adaptability by behavior selection and observation for mobile robots. In Soft Computing in Engineering Design and Manufacturing, R. Roy, P. Chawdry, and P. Pants (Eds.), Springer-Verlag.

  • Michaud, F., Lachiver, G., and Dinh, C.T.L. 1996. A new control architecture combining reactivity, deliberation and motivation for situated autonomous agent. In From Animals to Animats: Proc. 4th Int'l Conf. on Simulation of Adaptive Behavior, P. Maes, M.J. Matarić, J.-A. Meyer, J. Pollack, and S.W. Wilson (Eds.), Cape Cod, The MIT Press, pp. 247-254.

    Google Scholar 

  • Michaud, F. and Matarić, M.J. 1997. A history-based learning approach for adaptive robot behavior selection. Technical Report CS-97-192, Computer Science Department, Volen Center for Complex System, Brandeis University, Waltham, MA, USA.

    Google Scholar 

  • Ram, A. and Santamaria, J.C. 1993. Multistrategy learning in reactive control systems for autonomous robotic navigation. Informatica, 17(4):347-369.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Université de Sherbrooke, Sherbrooke, Québec, Canada, J1K 2R1

    François Michaud

  2. Computer Science Department, University of Southern California, Los Angeles, CA, 90089-0781

    Maja J. Matarić

Authors
  1. François Michaud
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Maja J. Matarić
    View author publications

    You can also search for this author inPubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Michaud, F., Matarić, M.J. Learning from History for Behavior-Based Mobile Robots in Non-Stationary Conditions. Autonomous Robots 5, 335–354 (1998). https://doi.org/10.1023/A:1008814507256

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008814507256