Abstract
Autonomous agents commonly suffer from perceptual aliasing in which differing situations are perceived as identical by the robots sensors, yet require different courses of action. One technique for addressing this problem is to use additional internal states within a reinforcement learning system, in particular a learning classifier system. Previous research has shown that adding internal memory states can allow an animat within a cellular world to successfully navigate complex mazes. However, the technique has not previously been applied to robotic environments in which sensory data is noisy and somewhat unpredictable. We present results of using XCS with additional internal memory in the simulated Khepera environment, and show that control rules can be evolved to allow the robot to navigate a variety of problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Maes, P., Brooks, R.: Learning to coordinate behaviors. In: Proceedings of the Eighth International Conference on Artificial Intelligence (AAAI 1990), pp. 796–802 (1990)
Lee, W.P.e.: Applying genetic programming to evolve behaviour primitives and arbitrators for mobile robots. In: Proceedings of the IEEE International Conference on Evolutionary Computation, Indianapolis, U A (2000)
Sutton, R.S.: Planning by incremental dynamic programming. In: Proceedings of the Eighth International Workshop on Machine Learning, pp. 353–357 (1991)
Chapman, D., Kaelbling, L.P.: Learning from delayed reinforcement ina complex domain. In: Proceedings of the 12th Int. Joint Conf. on Artificial Intelligence (1991)
Mahadevan, S., Connell, J.: Scaling reinforcement learning to robotics by exploting the subsumption arc itecture. In: Proceedings of the Eighth International Workshop on Machine Learning (1991)
Whitehead, S., Ballad, D.H.: Learning to perceive and act by trial and error. Machine Learning 7, 45–83 (1991)
McCallum, A.: Reinforcement Learning with Selective Perception and Hidden State. PhD thesis, University of Rochester (1996)
Sondik, E.: The optimal control of partially observable Markov processes. PhD thesis, Computer Science, Stanford University (1971)
Hansen, E.: Finite-memory control of partially observable systems. PhD thesis, Computer Science, University of Massachussetts at Amherst (1998)
Kim, D., Hallam, J.: An evolutionary approach to quantify internal states needed for the woods proble. In: Proceedings of the Seventh International Conference on the Simulation of Adaptive Behavior, MIT Press, From Animals to Animats (2000)
Wilson, S.: Zcs: a zeroth level classifier. Evolutionary Computation 2, 1–18 (1994)
Cliff, D., Ross, S.: Adding temporary memory to zcs. Adaptive Behavior 3, 101–150 (1994)
Lanzi, P.L.: Adding memory to xcs. In: Proceedings of the IEEE World Congress on Computational Intelligence, IEEE Press, Anchorage, Alaska, pp. 609–661 (1998)
Lanzi, P.L.: An analysis of the memory mechanism of XCSM. In: Genetic Programming 1998: Proceedings of the Third Annual Conference, pp. 643–665. Morgan Kaufmann, San Francisco (1998)
Wilson, S.W.: Generalization in xcs. Evolutionary Computation 3, 149–175 (1995)
Lanzi, P.: Adding memory to wilson–s xcs classifier system: to learn in partially observable environments. In: Procedings of AAAI Fall Symposium on Partiallyobservable Markov Decision Processes, pp. 91–98. AAAI Press, Menlo Park (1998)
Stolzmann, W., Butz, M.: Latent learning and action-planning in robots with anticipatory classifier systems. In: Lanzi, P.L., Stolzmann, W., Wilson, S. (eds.) Learning Classifier Systems: From Foundations to Application Advances in Evolutionary Computing, pp. 301–317. Springer, Heidelberg (2000)
Carse, B., Pipe, A.: X-fcs: A fuzzy classifier system using accuracy-based fitness - first results. Technical Report UWELCSG01-007, University of the West of England, Bristol (2001)
Hurst, J., Bull, L., Melhuish, C.: ZCS and TCS learning classifier system controllers on real robots. Technical Report UWELCSG02-002, University of the West of England, Bristol (2002)
Dorigo, M.: Alecsys and the autonomouse: Learning to control a real robot by distributed classifier systems. Machine Learning 19, 209–240 (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Webb, A., Hart, E., Ross, P., Lawson, A. (2003). Controlling a Simulated Khepera with an XCS Classifier System with Memory. In: Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., Kim, J.T. (eds) Advances in Artificial Life. ECAL 2003. Lecture Notes in Computer Science(), vol 2801. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39432-7_95
Download citation
DOI: https://doi.org/10.1007/978-3-540-39432-7_95
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20057-4
Online ISBN: 978-3-540-39432-7
eBook Packages: Springer Book Archive