Abstract
Due to their efficiency and adaptability, bio-inspired algorithms have shown their usefulness in a wide range of different non-linear optimization problems. In this paper, we compare two ways of training an artificial neural network (ANN): Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithms. The main contribution of this paper is to show which of these two algorithms provides the best accuracy during the learning phase of an ANN. First of all, we explain how the ANN training phase could be seen as an optimization problem. Then, we explain how PSO and DE could be applied to find the best synaptic weights of the ANN. Finally, we perform a comparison between PSO and DE approaches when used to train an ANN applied to different non-linear problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tejen, S., Jyunwei, J., Chengchih, H.: A Hybrid Artificial Neural Networks and Particle Swarm Optimization for Function Approximation. International Journal of Innovative Computing, Information and Control ICIC 4, 2363–2374 (2008)
Chau, K.W.: Application of a PSO-based neural network in analysis of outcomes of construction claims. Automation in Construction 16, 642–646 (2007)
Chatterjee, A., et al.: A Particle Swarm Optimized Fuzzy-Neural Network for Voice Controlled Robot Systems. IEEE Trans. on Ind. Elec. 52, 1478–1489 (2005)
Wang, Z., et al.: Particle Swarm Optimization and Neural Network Application for QSAR. In: Proceedings 18th Parallel and Distributed Processing Symposium (2004)
Zhao, L., Yang, Y.: PSO-Based Single Multiplicative Neuron Model for Time Series Prediction. Expert Systems with Applications 36, 2805–2812 (2009)
Da, Y., Ge, X.R.: An improved PSO–based ANN with simulated annealing technique. Neurocomput. Lett. 63, 527–533 (2005)
Yu, J., et al.: An Improved Particle Swarm Optimization for Evolving Feedforward Artificial Neural Networks. Neural Processing Letters 26, 217–231 (2007)
Mohaghegi, S., et al.: A comparison of PSO and backpropagation for training RBF neural networks for identification of a power system with STATCOM. In: Proceedings 2005 IEEE Swarm Intelligence Symposium on SIS 2005, pp. 381–384 (2005)
Ilonen, J., Kamarainen, J.-K., Lampinen, J.: Differential Evolution Training Algorithm for Feed-Forward Neural Networks. Neural Processing Letters 17(1), 93–105 (2003)
Castillo, P.A., Carpio, J., Merelo, J.J., Prieto, A., Rivas, V.: Evolving multilayer perceptrons. Neural Process. Lett. 12, 115–127 (2000)
Rivero, D., Rabuñal, J.R., Dorado, J., Pazos, A.: Automatic design of aNNs by means of GP for data mining tasks: Iris flower classification problem. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds.) ICANNGA 2007. LNCS, vol. 4431, pp. 276–285. Springer, Heidelberg (2007)
Yao, X.: Evolving Artificial Neural Networks. Proc. of IEEE 87(9), 1423–1446 (1999)
Kennedy, J., Eberhart, R.C., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)
Storn, R., Price, K.: Differential Evolution - a Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces, TR-95-012, ICSI (1995)
Mezura-Montes, E., Velazquez-Reyes, J., Coello Coello, C.A.: A Comparative Study of Differential Evolution Variants for Global Optimization. In: GECCO (2006)
Murphy, P.M., Aha, D.W.: UCI repository of machine learning databases. Dept. Inf. Comput. Sci., Univ. California, Irvine, CA (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Garro, B.A., Sossa, H., Vázquez, R.A. (2011). Evolving Neural Networks: A Comparison between Differential Evolution and Particle Swarm Optimization. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds) Advances in Swarm Intelligence. ICSI 2011. Lecture Notes in Computer Science, vol 6728. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21515-5_53
Download citation
DOI: https://doi.org/10.1007/978-3-642-21515-5_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21514-8
Online ISBN: 978-3-642-21515-5
eBook Packages: Computer ScienceComputer Science (R0)