Abstract
In this contribution we present an extension of a neuroevolutionary method called Evolutionary Acquisition of Neural Topologies (EANT) [11] that allows the evolution of solutions taking the form of a POMDP agent (Partially Observable Markov Decision Process) [8]. The solution we propose involves cascading a Kalman filter [10] (state estimator) and a feed-forward neural network. The extension (EANT+KALMAN) has been tested on the double pole balancing without velocity benchmark, achieving significantly better results than the to date published results of other algorithms.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Dürr, P., Mattiussi, C., Floreano, D.: Neuroevolution with analog genetic encoding. In: Proceedings of the 9th Conference on Parallel Problem Solving from Nature (PPSN IX), pp. 671–680 (2006)
Gomez, F.J., Miikkulainen, R.: Incremental evolution of complex general behavior. Adaptive Behavior 5, 317–342 (1997)
Gomez, F.J., Schmidhuber, J., Miikkulainen, R.: Efficient non-linear control through neuroevolution. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 654–662. Springer, Heidelberg (2006)
Gray, J.E., Murray, W.: A derivation of an analytic expression for the tracking index for alpha-beta-gamma filter. IEEE Transactions on Aerospace and Electronic Systems 29(3), 1064–1065 (1993)
Gruau, F.: Neural Network Synthesis Using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon, Laboratoire de l’Informatique du Parallelisme, France (January 1994)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)
Igel, C.: Neuroevolution for reinforcement learning using evolution strategies. In: Sarker, R., Reynolds, R., Abbass, H., Tan, K.C., McKay, B., Essam, D., Gedeon, T. (eds.) Congress on Evolutionary Computation (CEC 2003), vol. 4, pp. 2588–2595. IEEE Computer Society Press, Los Alamitos (2003)
Kaelbling, L.P., Littman, M.L., Moore, A.P.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Kalata, P.R.: Alpha-beta target tracking systems: A survey. In: American Control Conference, pp. 832–836 (1992)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, D35–45 (1960)
Kassahun, Y.: Towards a Unified Approach to Learning and Adaptation. PhD thesis, Technical Report 0602, Institute of Computer Science and Applied Mathematics, Christian-Albrechts University, Kiel, Germany (February 2006)
Kassahun, Y., de Gea, J., Edgington, M., Metzen, J., Kirchner, F.: Accelerating neuroevolutionary methods using a Kalman filter. In: Proceedings of the 10th Genetic and Evolutionary Computation Conference (GECCO 2008) (accepted, 2008)
Kassahun, Y., Edgington, M., Metzen, J.H., Sommer, G., Kirchner, F.: A common genetic encoding for both direct and indirect encodings of networks. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 1029–1036 (2007)
Stanley, K.O.: Efficient Evolution of Neural Networks through Complexification. PhD thesis, Artificial Intelligence Laboratory. The University of Texas at Austin, Austin, USA (August 2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kassahun, Y., de Gea, J., Metzen, J.H., Edgington, M., Kirchner, F. (2008). EANT+KALMAN: An Efficient Reinforcement Learning Method for Continuous State Partially Observable Domains. In: Dengel, A.R., Berns, K., Breuel, T.M., Bomarius, F., Roth-Berghofer, T.R. (eds) KI 2008: Advances in Artificial Intelligence. KI 2008. Lecture Notes in Computer Science(), vol 5243. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85845-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-540-85845-4_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85844-7
Online ISBN: 978-3-540-85845-4
eBook Packages: Computer ScienceComputer Science (R0)