Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Schuman, Catherine; Bruer, Grant; Young, Aaron; Dean, Mark; Plank, James

doi:10.1109/IJCNN.2018.8489149

Title: Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Conference · Mon Oct 01 00:00:00 EDT 2018

DOI:https://doi.org/10.1109/IJCNN.2018.8489149· OSTI ID:1479770

^[1]; Bruer, Grant ^[2]; Young, Aaron ^[2]; Dean, Mark ^[2]; Plank, James ^[2]

ORNL
University of Tennessee (UT)

Evolutionary optimization or genetic algorithms have been used to optimize a variety of neural network types, including spiking recurrent neural networks, and are attractive for many reasons. However, a key impediment to their widespread use is the potential for slow training times and failure to converge to a good fitness value in a reasonable amount of time. In this work, we evaluate the effect of different selection algorithms on the performance of an evolutionary optimization method for designing spiking recurrent neural networks, including those that are meant to be deployed in a neuromorphic system. We propose a selection approach that utilizes a richer understanding of the fitness of an individual network to inform the selection process and to promote diversity in the population. We show that including this feature can provide a significant increase in performance over utilizing a standard selection approach.

View Conference

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1479770

Resource Relation:: Conference: International Joint Conference on Neural Networks (IJCNN) - Rio de Janeiro, , Brazil - 7/8/2018 8:00:00 AM-7/13/2018 8:00:00 AM

Country of Publication:: United States

Language:: English

References (23)

A VLSI Implementation of an Analog Neural Network Suited for Genetic Algorithms Schemmel, Johannes; Meier, Karlheinz; Schürmann, Felix Evolvable Systems: From Biology to Hardware https://doi.org/10.1007/3-540-45443-8_5	book	January 2001
Evolutionary Multi-objective Optimization of Spiking Neural Networks Jin, Yaochu; Wen, Ruojing; Sendhoff, Bernhard Lecture Notes in Computer Science https://doi.org/10.1007/978-3-540-74690-4_38	book	January 2007
NeoN: Neuromorphic control for autonomous robotic navigation Mitchell, J. Parker; Bruer, Grant; Dean, Mark E. 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS) https://doi.org/10.1109/IRIS.2017.8250111	conference	October 2017
Spatiotemporal Classification Using Neuroscience-Inspired Dynamic Architectures Schuman, Catherine D.; Birdwell, J. Douglas; Dean, Mark E. Procedia Computer Science, Vol. 41 https://doi.org/10.1016/j.procs.2014.11.089	journal	January 2014
Unsupervised learning of digit recognition using spike-timing-dependent plasticity Diehl, Peter U.; Cook, Matthew Frontiers in Computational Neuroscience, Vol. 9 https://doi.org/10.3389/fncom.2015.00099	journal	August 2015
Mutation-Based Genetic Neural Network Palmes, P. P.; Hayasaka, T.; Usui, S. IEEE Transactions on Neural Networks, Vol. 16, Issue 3 https://doi.org/10.1109/TNN.2005.844858	journal	May 2005
Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke Kasabov, Nikola; Feigin, Valery; Hou, Zeng-Guang Neurocomputing, Vol. 134 https://doi.org/10.1016/j.neucom.2013.09.049	journal	June 2014
A Digital Liquid State Machine With Biologically Inspired Learning and Its Application to Speech Recognition Zhang, Yong; Li, Peng; Jin, Yingyezhe IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, Issue 11 https://doi.org/10.1109/TNNLS.2015.2388544	journal	November 2015
Design space exploration and parameter tuning for neuromorphic applications Carlson, Kristofor D.; Nageswaran, Jayram M.; Dutt, Nikil 2013 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS) https://doi.org/10.1109/CODES-ISSS.2013.6659007	conference	September 2013
Evolving artificial neural networks Proceedings of the IEEE, Vol. 87, Issue 9 https://doi.org/10.1109/5.784219	journal	January 1999
The Super-Turing Computational Power of Plastic Recurrent Neural Networks Cabessa, JÉRÉMie; Siegelmann, Hava T. International Journal of Neural Systems, Vol. 24, Issue 08 https://doi.org/10.1142/S0129065714500294	journal	November 2014
Networks of spiking neurons: The third generation of neural network models Maass, Wolfgang Neural Networks, Vol. 10, Issue 9 https://doi.org/10.1016/S0893-6080(97)00011-7	journal	December 1997
Error-backpropagation in temporally encoded networks of spiking neurons Bohte, Sander M.; Kok, Joost N.; La Poutré, Han Neurocomputing, Vol. 48, Issue 1-4 https://doi.org/10.1016/S0925-2312(01)00658-0	journal	October 2002
A Unified Hardware/Software Co-Design Framework for Neuromorphic Computing Devices and Applications Plank, James S.; Rose, Garrett S.; Dean, Mark E. 2017 IEEE International Conference on Rebooting Computing (ICRC) https://doi.org/10.1109/ICRC.2017.8123655	conference	November 2017
Spike timing dependent plasticity based enhanced self-learning for efficient pattern recognition in spiking neural networks Srinivasan, Gopalakrishnan; Roy, Sourjya; Raghunathan, Vijay 2017 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2017.7966075	conference	May 2017
Evolving Neural Networks through Augmenting Topologies Stanley, Kenneth O.; Miikkulainen, Risto Evolutionary Computation, Vol. 10, Issue 2 https://doi.org/10.1162/106365602320169811	journal	June 2002
Evolutionary reinforcement learning of artificial neural networks Siebel, Nils T.; Sommer, Gerald International Journal of Hybrid Intelligent Systems, Vol. 4, Issue 3 https://doi.org/10.3233/HIS-2007-4304	journal	October 2007
Towards evolving spiking networks with memristive synapses Howard, Gerard; Gale, Ella; Bull, Larry 2011 IEEE Symposium on Artificial Life (ALIFE) https://doi.org/10.1109/ALIFE.2011.5954655	conference	April 2011
Parallel Evolutionary Optimization for Neuromorphic Network Training Schuman, Catherine D.; Disney, Adam; Singh, Susheela P. 2016 2nd Workshop on Machine Learning in HPC Environments (MLHPC) https://doi.org/10.1109/MLHPC.2016.008	conference	November 2016
Neuroevolution: from architectures to learning Floreano, Dario; Dürr, Peter; Mattiussi, Claudio Evolutionary Intelligence, Vol. 1, Issue 1 https://doi.org/10.1007/s12065-007-0002-4	journal	January 2008
Movement prediction from real-world images using a liquid state machine Burgsteiner, Harald; Kröll, Mark; Leopold, Alexander Applied Intelligence, Vol. 26, Issue 2 https://doi.org/10.1007/s10489-006-0007-1	journal	November 2006
Evolving spiking neural networks for robot control Batllori, R.; Laramee, C. B.; Land, W. Procedia Computer Science, Vol. 6 https://doi.org/10.1016/j.procs.2011.08.060	journal	January 2011
An evolutionary optimization framework for neural networks and neuromorphic architectures Schuman, Catherine D.; Plank, James S.; Disney, Adam 2016 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2016.7727192	conference	July 2016

Similar Records

Multi-Objective Hyperparameter Optimization for Spiking Neural Network Neuroevolution

Conference · Tue Jun 01 00:00:00 EDT 2021 · OSTI ID:1479770

Parsa, Maryam; Kulkarni, Shruti; Coletti, Mark; +3 more

Multi-Objective Optimization for Size and Resilience of Spiking Neural Networks

Conference · Tue Oct 01 00:00:00 EDT 2019 · OSTI ID:1479770

Dimovska, Mihaela; Johnston, Travis; Schuman, Catherine; +2 more

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

Journal Article · Fri Jun 11 00:00:00 EDT 2021 · Neural Computation · OSTI ID:1479770

Zhang, Wenrui; Li, Peng

Title: Understanding Selection And Diversity For Evolution Of Spiking Recurrent Neural Networks

Citation Formats

References (23)

Similar Records

Related Subjects