Skip to main content
SpringerLink
Log in

Studying the Effect of Optimizing Weights in Neural Networks with Meta-Heuristic Techniques

  • Conference paper
  • First Online:
Proceedings of the International Conference on Data Engineering 2015 (DaEng-2015)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 520))

Abstract

Meta-heuristic algorithms provide derivative-free solutions to optimize complex problems. Back-propagation Neural Network (BP) algorithm is one of the most commonly used and a popular technique to optimize the feed forward neural network training. Traditional BP algorithm has some drawbacks, such as getting stuck easily in local minima and slow speed of convergence. This paper proposed a new meta-heuristic search algorithm, called cuckoo search (CS), based on cuckoo bird’s behavior to train back propagation (BP), Elman Recurrent Neural Network (RNN), and Levenberg Marquardt (LM) algorithms to achieving fast convergence rate and to avoid local minima problem. The performances of the proposed hybrid Cuckoo Search algorithms are compared with artificial bee colony using BP algorithm, and other hybrid variant. Specifically on Iris and 7-Bit parity datasets are used. The simulation results show that the hybrid Cuckoo Search show better performances than the other hybrid technique.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Radhika, Y., Shashi, M.: Atmospheric temperature prediction using support vector machines. Int. J. Comput. Theor. Eng. 1(1), 1793–8201 (2009)

    Google Scholar 

  2. Akcayol, M.A., Cinar, C.: Artificial neural network based modeling of heated catalytic converter performance. Appl. Therm. Eng. 25(14–15), 2341–2350 (2005)

    Article  Google Scholar 

  3. Shereef K.I., Baboo, S.S.: A new weather forecasting technique using back propagation neural network with modified Levenberg-Marquardt algorithm for learning. IJCSI Int. J. Comput. Sci. 8(6), 1694–0814 (2011) (No. 2)

    Google Scholar 

  4. Kosko, B.: Neural Network and Fuzzy Systems, 1st edn. Prentice Hall of India (1994)

    Google Scholar 

  5. Krasnopolsky, V.M., Chevallier, F.: Some neural network application in environmental sciences. Part II: Advancing computational efficiency of environmental numerical models. In: Neural Networks (eds.), vol. 16(3–4), pp. 335–348 (2003)

    Google Scholar 

  6. Coppin, B.: Artificial Intelligence Illuminated, pp. 291–324. Jones and Bartlet Illuminated Series, USA (2004)

    Google Scholar 

  7. Basheer, I.A., Hajmeer, M.: Artificial neural networks: fundamentals, computing, design, and application. J. Microbiol. Methods 43(1), 03–31 (2000)

    Article  Google Scholar 

  8. Zheng, H., Meng, W., Gong, B.: Neural Network and its Application on Machine fault Diagnosis. In: ICSYSE, pp. 576–579 (1992)

    Google Scholar 

  9. Rehman, M.Z, Nawi, N.M: Improving the accuracy of gradient descent back propagation algorithm (GDAM) on classification problems. Int. J. New Comput. Architectures Appl. (IJNCAA) The Society of Digital Information and Wireless Communication 1(4), 838–847 (2012)

    Google Scholar 

  10. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  11. Lahmiri, S.: A comparative study of backpropagation algorithms in financial prediction. Int. J. Comput. Sci. Eng. Appl. (IJCSEA) 1(4), 15–21 (2011)

    Google Scholar 

  12. Nawi, M.N. Ransing, R.S, Abdul Hamid, N.: BPGD-AG: a new improvement of back-propagation neural network learning algorithms with adaptive gain. J. Sci. Technol. 2(2) (2011)

    Google Scholar 

  13. Ahmed, W.A.M., Saad, E.S.M., Aziz, E.S.A.: Modified back propagation algorithm for learning artificial neural networks. In: Proceedings of the Eighteenth National Radio Science Conference (NRSC), pp. 345–352 (2001)

    Google Scholar 

  14. Wen, J., Zhao, J.L., Luo, S.W., Han, Z.: The improvements of BP neural network learning algorithm. In: Proceedings of the 5th International Conference on Signal Processing WCCC-ICSP, vol. 3, pp. 1647–1649 (2000)

    Google Scholar 

  15. Lahmiri, S.: Wavelet transform, neural networks and the prediction of s & p price index: a comparative paper of back propagation numerical algorithms. Bus. Intell. J. 5(2), 235 (2012)

    Google Scholar 

  16. Nawi, N.M., Ransing, R.S., Salleh, M.N.M., Ghazali, R., Hamid, N.A.: An improved back propagation neural network algorithm on classification problems. In: Communications in Computer and Information Science, vol. 118 CCIS, pp. 177–188 (2010)

    Chapter  Google Scholar 

  17. Gupta, J.N.D., Sexton, R.S.: Comparing back propagation with a genetic algorithm for neural network training. Int. J. Manage. Sci. 27, 679–684 (1999)

    Google Scholar 

  18. Nawi, N.M., Ghazali, R., Salleh, M.N.M.: The development of improved back-propagation neural networks algorithm for predicting patients with heart disease. In: LNCS, vol. 6377, no. 4, pp. 317–324 (2010)

    Google Scholar 

  19. Yam, J.Y.F., Chow, T.W.S.: Extended least squares based algorithm for training feed forward networks. IEEE Trans. Neural Netw. 8, 806–810 (1997)

    Article  Google Scholar 

  20. Yam, J.Y.F., Chow, T.W.S.: A weight initialization method for improving training speed in feed forward neural networks. Neurocomputing 30, 219–232 (2000)

    Article  Google Scholar 

  21. Yam, J.Y.F., Chow, T.W.S.: Feed forward networks training speed enhancement by optimal initialization of the synaptic coefficients. IEEE Trans. Neural Netw. 12, 430–434 (2001)

    Article  Google Scholar 

  22. Kwok, T.Y., Yeung, D.Y.: Objective functions for training new hidden units in constructive neural networks. IEEE Trans. Neural Netw. 8, 1131–1147 (1997)

    Article  Google Scholar 

  23. Zhang, J. R., Zhang, J., Lok, T., Lyu, M.: A hybrid particle swarm optimization back propagation algorithm for feed forward neural network training. Appl. Math. Comput. 185(5), 1026–1037 (ELSEVIER) (2007)

    Article  Google Scholar 

  24. Shah, H., Ghazali, R., Nawi, N.M., Deris, M.M.: Global hybrid ant bee colony algorithm for training artificial neural networks. LNCS, vol. 7333, no. 1, Pages 87–100 (2012)

    Google Scholar 

  25. Chiroma, H., Abdul-Kareem, S., Khan, A., Nawi, N.M., Ya’U Gital, A., Shuib, L., AbuBakar, A.I., Rahman, M.Z., Herawan, T.: Global warming: predicting OPEC carbon dioxide emissions from petroleum consumption using neural network and hybrid cuckoo search algorithm. PLoS ONE 10(8), 25 (2015)

    Article  Google Scholar 

  26. Karaboga, D.: Artificial bee colony algorithm. Scholarpedia 5(3), 6915 (2010)

    Article  Google Scholar 

  27. Yao, X.: Evolutionary artificial neural networks. Int. J. Neural Syst. 4(3), 203–222 (1993)

    Article  Google Scholar 

  28. Mendes, R., Cortez, P., Rocha, M., Neves, J.: Particle swarm for feedforward neural network training. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 1895–1899 (2002)

    Google Scholar 

  29. Lonen, J., Kamarainen, J.I., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. J. Neural Process. Lett. 17(1), 93–105 (2003)

    Article  Google Scholar 

  30. Liu, Y.-P., M.-G. Wu, et al.: Evolving neural networks using the hybrid of ant colony optimization and BP algorithms. In: Wang, J., Yi, Z., Zurada, J.M., Lu, B.-L., Yin, H. (eds.) Advances in Neural Networks, ISNN 2006, vol. 3971, pp. 714–722. Springer Berlin/Heidelberg (2006)

    Chapter  Google Scholar 

  31. Khan, A.U., Bandopadhyaya, T.K., Sharma, S.: Comparisons of stock rates prediction accuracy using different technical indicators with back propagation neural network and genetic algorithm based back propagation neural network. In: The Proceedings of the First International Conference on Emerging Trends in Engineering and Technology IEEE Computer Society, Nagpur, India, pp. 575–580 (2008)

    Google Scholar 

  32. Yang, X.S., Deb, S.: Cuckoo search via Lévy flights. In: Proceedings of World Congress on Nature & Biologically Inspired Computing, India, pp. 210–214 (2009)

    Google Scholar 

  33. Yang, X.S., Deb, S.: Engineering optimisation by cuckoo search. Int. J. Math. Model. Numer. Optimisation 1(4), 330–343 (2010)

    Article  Google Scholar 

  34. Tuba, M., Subotic, M., Stanarevic, N.: Modified cuckoo search algorithm for unconstrained optimization problems. In: Proceedings of the European Computing Conference, pp. 263–268. Paris, France (2011)

    Google Scholar 

  35. M. Tuba, M. Subotic, and N. Stanarevic Performance of a Modified Cuckoo Search Algorithm for Unconstrained Optimization Problems Faculty of Computer Science Faculty of Computer Science, vol. 11, no. 2, pp. 62–74 (2012)

    Google Scholar 

  36. Nawi, N.M., Khan, A., Rehman, M.: CSLM: Levenberg Marquardt based Back Propagation Algorithm Optimized with Cuckoo Search (2013)

    Google Scholar 

  37. Nawi, N.M., Khan, A., Rehman, M.Z.: A New Optimized Cuckoo Search Recurrent Neural Network (CSRNN) Algorithm. In: The 8th International Conference on Robotic, Vision, Signal Processing & Power Applications, pp. 335–341 (2014)

    Google Scholar 

  38. Nawi, N.M., Khan, A., Rehman, M.Z.: A new back-propagation neural network optimized with cuckoo search algorithm. In: Computational Science and Its Applications–ICCSA 2013, pp. 413–426. Springer Berlin Heidelberg (2013)

    Google Scholar 

  39. Chaowanawate, K., Heednacram, A.: Implementation of cuckoo search in RBF neural network for flood forecasting. In: Fourth International Conference on Computational Intelligence, Communication Systems and Networks, pp. 22–26 (2012)

    Google Scholar 

  40. Pavlyukevich, I.: Levy flights, non-local search and simulated annealing. J. Comput. Phys. 226(2), 1830–1844 (2007)

    Article  MathSciNet  Google Scholar 

  41. Walton, S., Hassan, O., Morgan, K., Brown, M.: Modified cuckoo search: a new gradient free optimisation algorithm. J. Chaos, Solitons Fractals 44(9), 710–718 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Universiti Tun Hussein Onn Malaysia (UTHM) Ministry of Higher Education (MOHE) Malaysia for financially supporting this Research under Trans-disciplinary Research Grant Scheme (TRGS) vote no. T003. This research also supported by GATES IT Solution Sdn. Bhd under its publication scheme.

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, Soft Computing and Data Mining Centre, Universiti Tun Hussein Onn Malaysia (UTHM), 101, 86400, Parit Raja, Batu Pahat, Johor, Malaysia

    Nazri Mohd. Nawi & Jamal Uddin

  2. Institute of Business and Management Science, Agriculture University Peshawar, Peshawar, Pakistan

    Abdullah Khan

  3. Department of Computer Science and Information Technology, University of Lahore, Islamabad Campus, Japan Road, Sihala, Pakistan

    M. Z. Rehman & Rashid Naseem

Authors
  1. Nazri Mohd. Nawi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdullah Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Z. Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rashid Naseem
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jamal Uddin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nazri Mohd. Nawi .

Editor information

Editors and Affiliations

  1. School of Information Technology, Deakin University, Geelong, VIC, Australia

    Jemal H. Abawajy

  2. Department of Communication Technology and Network, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Malaysia

    Mohamed Othman

  3. Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, Malaysia

    Rozaida Ghazali

  4. Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, Malaysia

    Mustafa Mat Deris

  5. Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, Batu Pahat, Johor, Malaysia

    Hairulnizam Mahdin

  6. Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Tutut Herawan

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nawi, N.M., Khan, A., Rehman, M.Z., Naseem, R., Uddin, J. (2019). Studying the Effect of Optimizing Weights in Neural Networks with Meta-Heuristic Techniques. In: Abawajy, J., Othman, M., Ghazali, R., Deris, M., Mahdin, H., Herawan, T. (eds) Proceedings of the International Conference on Data Engineering 2015 (DaEng-2015) . Lecture Notes in Electrical Engineering, vol 520. Springer, Singapore. https://doi.org/10.1007/978-981-13-1799-6_34

Download citation

Publish with us

Policies and ethics

Search

Navigation