Abstract
Multilayer perceptron networks have been successfully trai- ned by error backpropagation algorithm. We show that Particle Swarm Optimization(PSO) with minimum velocity constraints can efficiently be applied to train multilayer perceptrons to overcome premature convergence and alleviates the influence of dimensionality increasing. The experiments of two multilayer perceptrons trained by PSO with minimum velocity constraints are carried out. The result clearly demonstrate the improvement of the proposed algorithm over the standard PSO in terms of convergence.
This work was supported by National Science Foundation of China under Grant 60471055, Specialized Research Fund for the Doctoral Program of Higher Education under Grant 20040614017, and the 2006 Youth Science and Technology Key Foundation of UESTC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Haykin, S.: Neural Networks-A Comprehensive Foundation, 2nd edn. Tsinghua University Press, Beijing (2002)
Werbos, P.J.: Backpropagation through time: what it does and how to do it. Proceedings of the IEEE 78, 1550–1560 (1990)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learnng representations by Back-propagating errors. Nature 323, 533–536 (1986)
Hornik, K.M., Stinchcombe, M., White, M.: Multilayer feedfoward networks are universal approximators. Neural Networks 2, 359–366 (1989)
Battiti, R.: First- and second-order methods of learning: betwwen steepest descent and Newton’s method. Neural Computation 4, 141–166 (1991)
Rosario, R.: A Conjugate-gradient based algrithm for training of the feed-forward neural networks. Ph.D. Dissertation, Melbourne: Florida Institute of Technology (September 1989)
Hagan, M.T., Menhaj, M.B.: Training feedfoward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5, 989–993 (1994)
Salerno, J.: Using the Particle Swarm Optimization Technique to Train a Recurrent Neural Model. In: Proceedings of the IEEE International Conference on Tools with Artificial Intelligence, pp. 45–49 (1997)
Zhang, C., Shao, H., Li, Y.: Particle swarm optimisation for evolving artificial neural netowrk. In: Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, vol. 4, pp. 2487–2490 (2000)
Settles, M., Rylander, B.: Neural Network Learning using Particle Swarm Optimizers. Advances in Information Science and Soft Computing, 224–226 (2002)
Gudise, V.G., Venayagamoothy, G.K.: Comparison of Particle Swarm optimization and backpropagation as training algorithms for neural networks. In: Proceedings of the IEEE Swarm Intelligence Symposium, Indiana, pp. 110–117 (2003)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proc. IEEE Intl. Conf. on Neural Networks IV, pp. 1942–1948. IEEE Service Center, Piscataway (1995)
Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, pp. 39–43. IEEE Service Center, Piscataway (1995)
Yasuda, K., Iwasaki, N.: Adaptive particle swarm optimization using velocity information of swarm. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, pp. 3475–3481 (2004)
Liu, H., Abraham, A.: Fuzzy Turbulent Particle Swarm Optimization. In: Proceeding of the 5th International Conference on Hybrid Intelligent Systems, Brazil, IEEE Press, Los Alamitos (2005)
Pohlheim, H.: Genetic Algorithm Toolbox Test Functions. Department of Automatic Control and systems Engineering University of Sheffield
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Pu, X., Fang, Z., Liu, Y. (2007). Multilayer Perceptron Networks Training Using Particle Swarm Optimization with Minimum Velocity Constraints. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds) Advances in Neural Networks – ISNN 2007. ISNN 2007. Lecture Notes in Computer Science, vol 4493. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72395-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-540-72395-0_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72394-3
Online ISBN: 978-3-540-72395-0
eBook Packages: Computer ScienceComputer Science (R0)