Abstract
A novel method is presented to allow a machine controller to evolve while the machine is acting in its environment. The method uses a single spiking neural network with a minimum number of neurons and no initial connections. New connections and neurons are grown by evaluating reward values which can represent either the internal state of the machine or the rating of its task performance. This way the topology and the level of connectivity of the network are kept to a minimum. The method will be applied to a controller for an autonomous mobile robot.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alnajjar, F., Murase, K.: Self-organization of Spiking Neural Network Generating Autonomous Behavior in a Real Mobile Robot. In: Proceedings of the International Conference on Computational Intelligence for Modelling, Control and Automation, vol. 1, pp. 1134–1139 (2005)
Alnajjar, F., Bin Mohd Zin, I., Murase, K.: A Spiking Neural Network with Dynamic Memory for a Real Autonomous Mobile Robot in Dynamic Environment. In: 2008 International Joint Conference on Neural Networks (2008)
Dauc, E., Henry, F.: Hebbian Learning in Large Recurrent Neural Networks. Movement and Perception Lab, Marseille (2006)
Elizondo, D., Birkenhead, R., Taillard, E.: Generalisation and the Recursive Deterministic Perceptron. In: International Joint Conference on Neural Networks, pp. 1776–1783 (2006)
Elizondo, D., Fiesler, E., Korczak, J.: Non-ontogenetic Sparse Neural Networks. In: International Conference on Neural Networks, vol. 26, pp. 290–295. IEEE, Los Alamitos (1995)
Florian, R.V.: Reinforcement Learning Through Modulation of Spike-timing-dependent Synaptic Plasticity. Neural Computation 19(6), 1468–1502 (2007)
Fritzke, B.: Fast Learning with Incremental RBF Networks. Neural Processing Letters 1(1), 2–5 (1994)
Fritzke, B.: A Growing Neural Gas Network Learns Topologies. In: Advances in Neural Information Processing Systems, vol. 7, pp. 625–632 (1995)
Gómez, G., Lungarella, M., Hotz, P.E., Matsushita, K., Pfeifer, R.: Simulating Development in a Real Robot: On the Concurrent Increase of Sensory, Motor, and Neural Complexity. In: Proceedings of the Fourth International Workshop on Epigenetic Robotics, pp. 119–122 (2004)
Greenwood, G.W.: Attaining Fault Tolerance through Self-adaption: The Strengths and Weaknesses of Evolvable Hardware Approaches. In: Zurada, J.M., Yen, G.G., Wang, J. (eds.) WCCI 2008. LNCS, vol. 5050, pp. 368–387. Springer, Heidelberg (2008)
Hebb, D.O.: The Organization of Behaviour: A Neuropsychological Approach. John Wiley & Sons, New York (1949)
Izhikevich, E.M.: Which Model to Use for Cortical Spiking Neurons? IEEE Transactions on Neural Networks 15(5), 1063–1070 (2004)
Izhikevich, E.M.: Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cerebral Cortex 10, 1093–1102 (2007)
Jaeger, H.: The “Echo State” Approach to Analysing and Training Recurrent Neural Networks. GMD Report 148, German National Research Institute for Computer Science (2001)
Katic, D.: Leaky-integrate-and-fire und Spike Response Modell. Institut für Technische Informatik, Universität Karlsruhe (2006)
Kohonen, T.: Self-organization and Associative Memory. 3rd printing. Springer, Heidelberg (1989)
Liu, J., Buller, A.: Self-development of Motor Abilities Resulting from the Growth of a Neural Network Reinforced by Pleasure and Tension. In: Proceedings of the 4th International Conference on Development and Learning, pp. 121–125 (2005)
Liu, J., Buller, A., Joachimczak, M.: Self-motivated Learning Agent: Skill-development in a Growing Network Mediated by Pleasure and Tensions. Transactions of the Institute of Systems, Control and Information Engineers 19(5), 169–176 (2006)
Maass, W., Natschlaeger, T., Markram, H.: Real-time Computing without Stable States: A New Framework for Neural Computation Based on Perturbations. Neural Computation 14(11), 2531–2560 (2002)
Maes, P., Brooks, R.A.: Learning to Coordinate Behaviors. In: AAAI, pp. 796–802 (1990)
Martinetz, T.M., Schulten, K.J.: A “Neural-Gas” Network Learns Topologies. In: Kohonen, T., Mäkisara, K., Simula, O., Kangas, J. (eds.) Artificial Neural Networks, pp. 397–402 (1991)
Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. In: Adaptive Computation and Machine Learning. The MIT Press, Cambridge (2001)
Stanley, K.O., D’Ambrosio, D., Gauci, J.: A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks. Accepted to appear in Artificial Life journal (2009)
Tajine, M., Elizondo, D.: The Recursive Deterministic Perceptron Neural Network. Neural Networks 11, 1571–1588 (1998)
Vreeken, J.: Spiking Neural Networks, an Introduction. Intelligent Systems Group, Institute for Information and Computing Sciences, Utrecht University (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Huemer, A., Elizondo, D., Gongora, M. (2009). A Constructive Neural Network for Evolving a Machine Controller in Real-Time. In: Franco, L., Elizondo, D.A., Jerez, J.M. (eds) Constructive Neural Networks. Studies in Computational Intelligence, vol 258. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04512-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-04512-7_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04511-0
Online ISBN: 978-3-642-04512-7
eBook Packages: EngineeringEngineering (R0)