Abstract
Back propagation neural network (BPNN) algorithm is a widely used technique in training artificial neural networks. It is also a very popular optimization procedure applied to find optimal weights in a training process. However, traditional back propagation optimized with Levenberg marquardt training algorithm has some drawbacks such as getting stuck in local minima, and network stagnancy. This paper proposed an improved Levenberg-Marquardt back propagation (LMBP) algorithm integrated and trained with Cuckoo Search (CS) algorithm to avoided local minima problem and achieves fast convergence. The performance of the proposed Cuckoo Search Levenberg-Marquardt (CSLM) algorithm is compared with Artificial Bee Colony (ABC) and similar hybrid variants. The simulation results show that the proposed CSLM algorithm performs better than other algorithm used in this study in term of convergence rate and accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dayhoff, J.E.: Neural-Network Architectures: An Introduction, 1st edn. Van Nostrand Reinhold Publishers, New York (1990)
Mehrotra, K., Mohan, C., Ranka, S.: Elements of Artificial Neural Networks. MIT Press, Cambridge (1997)
Rehman, M.Z., Nazri, M.N.: Studying the Effect of adaptive momentum in improving the accuracy of gradient descent back propagation algorithm on classification problems. International Journal of Modern Physics (IJMPCS) 9(1), 432–439 (2012)
Ozturk, C., Karaboga, D.: Hybrid Artificial Bee Colony algorithm for neural network training. In: IEEE Congress of Evolutionary Computation (CEC), pp. 84–88 (2011)
Leung, C., Member, C.: A Hybrid Global Learning Algorithm Based on Global Search and Least Squares Techniques for back propagation neural network Networks. In: International Conference on Neural Networks, pp. 1890–1895 (1994)
Yao, X.: Evolutionary artificial neural networks. International Journal of Neural Systems 4(3), 203–222 (1993)
Mendes, R., Cortez, P., Rocha, M., Neves, J.: Particle swarm for feed forward neural network training. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1895–1899 (2002)
Nawi, N.M., Ghazali, R., Salleh, M.N.M.: The development of improved back-propagation neural networks algorithm for predicting patients with heart disease. In: Zhu, R., Zhang, Y., Liu, B., Liu, C. (eds.) ICICA 2010. LNCS, vol. 6377, pp. 317–324. Springer, Heidelberg (2010)
Abid, S., Fnaiech, F., Najim, M.: A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Transactions on NeuralNetworks 12, 424–430 (2001)
Yu, X., OnderEfe, M., Kaynak, O.: A general backpropagation algorithm for feedforward neural networks learning. IEEE Transactions on Neural Networks 13, 251–259 (2002)
Chronopoulos, A.T., Sarangapani, J.: A distributed discrete time neural network architecture forpattern allocation and control. In: Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS 2002), Florida, USA, pp. 204–211 (2002)
Wilamowski, B.M.: Neural Networks and Fuzzy Systems. CRC Press (2002)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Internal Representations by error Propagation. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1 (1986)
Gupta, J.N.D., Sexton, R.S.: Comparing backpropagation with a genetic algorithm for neural network training. The International Journal of Management Science 27, 679–684 (1999)
Salchenberger, L.M., Cinar, E.M., Lash, N.A.: Neural Networks: A New Tool for Predicting Thrift Failures. Decision Sciences 23(4), 899–916 (1992)
Sexton, R.S., Dorsey, R.E., Johnson, J.D.: Toward Global Optimization of Neural Networks: A Comparison of the Genetic Algorithm and Back propagation. Decision Support Systems 22, 171–186 (1998)
SNNS (Stuttgart Neural Network Simulator) (2013), http://wwwra.informatik.unituebingen.de/SNNS/ (Accessed February 2, 2013 )
Fahlman, S.E.: Faster-Learning Variations on Back-Propagation: An Empirical Study. In: Proceedings of the 1988 Connectionist Models Summer School. Morgan-Kaufmann, Los Altos (1988)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks, ICNN 1993 (1993)
Fahlman, S.E., Lebiere, C.: The Cascade-Correlation Learning Architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2. Morgan-Kaufmann, Los Altos (1990)
Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 23, 899–916 (1994)
Wilamowski, B.M., Cotton, N., Kaynak, O.: Neural Network Trainer with Second Order Learning Algorithms. In: 11th International Conference on Intelligent Engineering Systems, Budapest, Hungary (2007)
Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Trans. Neural Networks 5(6), 989–993 (1994)
Cao, X.P., Hu, C.H., Zheng, Z.Q., Lv, Y.J.: Fault Prediction for Inertial Device Based on LMBP Neural Network. Electronics Optics & Control 12(6), 38–41 (2005)
Haykin, S.: Neural Networks: A Comprehensive Foundation. China Machine Press, Beijing (2004)
Xue, Q., et al.: Improved LMBP Algorithm in the Analysis and Application of Simulation Data. In: 2010 International Conference on Computer Application and System Modeling (2010)
Yan, J., Cao, H., Wang, J., Liu, Y., Zhao, H.: Levenberg-Marquardt algorithm applied to forecast the ice conditions in Ningmeng Reach of the Yellow River. In: Fifth International Conference on Natural Computation (2009)
Yang, X.S., Deb, S.: Cuckoo search via Lévy flights. In: Proceedings of World Congress on Nature & Biologically Inspired Computing, India, pp. 210–214 (2009)
Yang, X.S., Deb, S.: Cuckoo search via levy flights. In: Nature Biologically Inspired Computing (NaBIC), pp. 210–214 (2009)
Deng, W.J., Chen, W.C., Pei, W.: Back-propagation neural network based importance-performance analysis for determining critical service attributes. Expert Systems with Applications 4(2) (2008)
Kosko, B.: Neural Network and Fuzzy Systems, 1st edn. Prentice Hall of India (1994)
Lee, T.: Back-propagation neural network for the prediction of the short-term storm surge in Taichung harbor, Taiwan. Engineering Applications of Artificial Intelligence 21(1) (2008)
Yang, X.S., Deb, S.: Engineering optimisation by Cuckoo Search. International Journal of Mathematical Modelling and Numerical Optimisation 1, 330–343 (2010)
Kumar, M.P.: Backpropagation Learning Algorithm based on Levenberg Marquardt Algorithm. In: Proceedings of CSCP, pp. 393–398 (2012)
Nourani, E., Rahmani, A.M., Navin, A.H.: Forecasting Stock Prices using a hybrid Artificial Bee Colony based Neural Network. In: International Conference on Innovation, Management and Technology Research (ICIMTR 2012), Malacca, Malaysia (2012)
Chaowanawatee, K., Heednacram, H.: Implementation of Cuckoo Search in RBF Neural Network for Flood Forecasting. In: Fourth International Conference on Computational Intelligence, Communication Systems and Networks, pp. 22–26 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nawi, N.M., Khan, A., Rehman, M.Z. (2013). A New Cuckoo Search Based Levenberg-Marquardt (CSLM) Algorithm. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2013. ICCSA 2013. Lecture Notes in Computer Science, vol 7971. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39637-3_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-39637-3_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39636-6
Online ISBN: 978-3-642-39637-3
eBook Packages: Computer ScienceComputer Science (R0)