Abstract
Recurrent Neural Networks (RNN) have local feedback loops inside the network which allows them to store earlier accessible patterns. This network can be trained with gradient descent back propagation and optimization technique such as second-order methods. Levenberg-Marquardt has been used for networks training but still this algorithm is not definite to find the global minima of the error function. Nature inspired meta-heuristic algorithms provide derivative-free solution to optimize complex problems. This paper proposed a new meta-heuristic search algorithm, called Cuckoo Search (CS) to train Levenberg Marquardt Elman Network (LMEN) in achieving fast convergence rate and to avoid local minima problem. The proposed Cuckoo Search Levenberg Marquardt Elman Network (CSLMEN) results are compared with Artificial Bee Colony using BP algorithm, and other hybrid variants. Specifically 7-bit parity and Iris classification datasets are used. The simulation results show that the computational efficiency of the proposed CSLMEN training process is highly enhanced when coupled with the Cuckoo Search method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Firdaus, A.A.M., Mariyam, S.H.S., Razana, A.: Enhancement of Particle Swarm Optimization in Elman Recurrent Network with bounded Vmax Function. In: Third Asia International Conference on Modelling & Simulation (2009)
Peng, X.G., Venayagamoorthy, K., Corzin, K.A.: Combined Training of Recurrent Neural Networks with Particle Swarm Optimization and Back propagation Algorithms for Impedance Identification. In: IEEE Swarm Intelligence Symposium, pp. 9–15 (2007)
Haykin, S.: Neural Networks:A Comprehensive Foundation, 2nd edn., pp. 84–89 (1999) ISBN 0-13-273350
Ubeyli, E.D.: Recurrent neural networks employing lyapunov exponents for analysis of Doppler ultrasound signals. J. Expert Systems with Applications 34(4), 2538–2544 (2008)
Ubeyli, E.D.: Recurrent neural networks with composite features for detection of electrocardiographic changes in partial epileptic patients. J. Computers in Biology and Medicine 38(3), 401–410 (2008)
Saad, E.W., Prokhorov, D.V., WunschII, D.C.: Comparative study of stock trend prediction using time delay, recurrent and probabilistic neural networks. J. IEEE Transactions on Neural Networks 9(6), 1456–1470 (1998)
Gupta, L., McAvoy, M.: Investigating the prediction capabilities of the simple recurrent neural network on real temporal sequences. J. Pattern Recognition 33(12), 2075–2081 (2000)
Gupta, L., McAvoy, M., Phegley, J.: Classification of temporal sequences via prediction using the simple recurrent neural network. J. Pattern Recognition 33(10), 1759–1770 (2000)
Nihal, G.F., Elif, U.D., Inan, G.: Recurrent neural networks employing lyapunov exponents for EEG signals classification. J. Expert Systems with Applications 29, 506–514 (2005)
Elman, J.L.: Finding structure in time. J. Cognitive Science 14(2), 179–211 (1990)
Rehman, M.Z., Nawi, N.M.: Improving the Accuracy of Gradient Descent Back Propagation Algorithm(GDAM) on Classification Problems. Int. J. of New Computer Architectures and their Applications (IJNCAA) 1(4), 838–847 (2012)
Wam, A., Esm, S., Esa, A.: Modified Back Propagation Algorithm for Learning Artificial Neural Networks. In: The 18th National Radio Science Conference, pp. 345–352 (2001)
Wen, J., Zhao, J.L., Luo, S.W., Han, Z.: The Improvements of BP Neural Network Learning Algorithm. In: 5th Int. Conf. on Signal Processing WCCC-ICSP, pp. 1647–1649 (2000)
Tanoto, Y., Ongsakul, W., Charles, O., Marpaung, P.: Levenberg-Marquardt Recurrent Networks for Long-Term Electricity Peak Load Forecasting. J. Telkomnika 9(2), 257–266 (2011)
Peng, C., Magoulas, G.D.: NonmonotoneLevenberg–Marquardt training of recurrent neural architectures for processing symbolic sequences. J. of Neural Comput& Application 20, 897–908 (2011)
Yang, X.S., Deb, S.: Cuckoo search via Lévy flights. In: Proceedings of World Congress on Nature & Biologically Inspired Computing, India, pp. 210–214 (2009)
Yang, X.S., Deb, S.: Engineering Optimisation by Cuckoo Search. J. International Journal of Mathematical Modelling and Numerical Optimisation 1(4), 330–343 (2010)
Tuba, M., Subotic, M., Stanarevic, N.: Modified cuckoo search algorithm for unconstrained optimization problems. In: Proceedings of the European Computing Conference (ECC 2011), pp. 263–268 (2011)
Tuba, M., Subotic, M., Stanarevic, N.: Performance of a Modified Cuckoo Search Algorithm for Unconstrained Optimization Problems. J. Faculty of Computer Science 11(2), 62–74 (2012)
Yang, X.S., Deb, S.: Engineering optimisation by cuckoo search. J. Int. J. Mathematical Modelling and Numerical Optimisation 1(4), 330–343 (2010)
Chaowanawate, K., Heednacram, A.: Implementation of Cuckoo Search in RBF Neural Network for Flood Forecasting. In: 4th International Conference on Computational Intelligence, Communication Systems and Networks, pp. 22–26 (2012)
Hagan, M.T., Menhaj, M.B.: Training Feedforward Networks with the Marquardt Algorithm. J. IEEE Transactions on Neural Networks 5(6), 989–993 (1999)
Nourani, E., Rahmani, A.M., Navin, A.H.: Forecasting Stock Prices using a hybrid Artificial Bee Colony based Neural Network. In: ICIMTR 2012, Malacca, Malaysia, pp. 21–22 (2012)
Rehman, M.Z., Nawi, N.M.: Studying the Effect of adaptive momentum in improving the accuracy of gradient descent back propagation algorithm on classification problems. J. Int. Journal of Modern Physics: Conference Series 9, 432–439 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Nawi, N.M., Khan, A., Rehman, M.Z., Herawan, T., Deris, M.M. (2014). CSLMEN: A New Cuckoo Search Levenberg Marquardt Elman Network for Data Classification. In: Herawan, T., Ghazali, R., Deris, M. (eds) Recent Advances on Soft Computing and Data Mining. Advances in Intelligent Systems and Computing, vol 287. Springer, Cham. https://doi.org/10.1007/978-3-319-07692-8_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-07692-8_17
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07691-1
Online ISBN: 978-3-319-07692-8
eBook Packages: EngineeringEngineering (R0)