Skip to main content
SpringerLink
Log in
Book cover

International Conference on Neural Information Processing

ICONIP 2014: Neural Information Processing pp 245–253Cite as

An Accelerated Particle Swarm Optimization Based Levenberg Marquardt Back Propagation Algorithm

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8835))

Abstract

The Levenberg Marquardt (LM) algorithm is one of the most effective algorithms in speeding up the convergence rate of the Artificial Neural Networks (ANN) with Multilayer Perceptron (MLP) architectures. However, the LM algorithm suffers the problem of local minimum entrapment. Therefore, we introduce several improvements to the Levenberg Marquardt algorithm by training the ANNs with meta-heuristic nature inspired algorithm. This paper proposes a hybrid technique Accelerated Particle Swarm Optimization using Levenberg Marquardt (APSO_LM) to achieve faster convergence rate and to avoid local minima problem. These techniques are chosen since they provide faster training for solving pattern recognition problems using the numerical optimization technique.The performances of the proposed algorithm is evaluated using some bench mark of classification’s datasets. The results are compared with Artificial Bee Colony (ABC) Algorithm using Back Propagation Neural Network (BPNN) algorithm and other hybrid variants.Based on the experimental result, the proposed algorithms APSO_LM successfully demonstrated better performance as compared to other existing algorithms in terms of convergence speed and Mean Squared Error (MSE) by introducing the error and accuracy in network convergence.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zheng, H., Meng, W., Gong, B.: Neural Network and its Application on Machine fault Diagnosis. In: ICSYSE 1992, pp. 576–579 (1992)

    Google Scholar 

  2. Kosko, B.: Neural Network and Fuzzy Systems, 1st edn. Prentice Hall of India (1992)

    Google Scholar 

  3. Basheer, I.A., Hajmeer: Artificial Neural Networks: fundamentals, computing, design and application. Journal of Microbiological Methods 43(1), 3–31 (2000)

    Article  Google Scholar 

  4. Krasnopolsky, V.M., Chevallier: Some Neural Network applications in environmental sciences. Part II, advancing computational efficiency of environmental numerical models. Neural Networks 16(3), 335–348 (2003)

    Article  Google Scholar 

  5. Coppin, B.: Artificial Intelligence Illuminated. Jones and Bartlet illuminated Series, USA, pp. 291–324 (2004)

    Google Scholar 

  6. Nawi, N.M., Khan, A., Rehman, M.Z.: A new cuckoo search based levenberg-marquardt (CSLM) algorithm. In: Murgante, B., Misra, S., Carlini, M., Torre, C.M., Nguyen, H.-Q., Taniar, D., Apduhan, B.O., Gervasi, O. (eds.) ICCSA 2013, Part I. LNCS, vol. 7971, pp. 438–451. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  7. Nawi, N.M., Khan, A., Rehman, M.Z.: A new back-propagation neural network optimized with cuckoo search algorithm. In: Murgante, B., Misra, S., Carlini, M., Torre, C.M., Nguyen, H.-Q., Taniar, D., Apduhan, B.O., Gervasi, O. (eds.) ICCSA 2013, Part I. LNCS, vol. 7971, pp. 413–426. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  8. Contreras, J., et al.: ARIMA models to predict next-day electricity prices. IEEE Transactions on Power Systems 18, 1014–1020 (2003)

    Article  Google Scholar 

  9. Leung, C.T., Chow, T.: A hybrid global learning algorithm based on global search and least squares techniques for back propagation networks. In: International Conference on Neural Networks, pp. 1890–1895 (1997)

    Google Scholar 

  10. Zhang, J.-R., et al.: A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Applied Mathematics and Computation 185, 1026–1037 (2007)

    Article  MATH  Google Scholar 

  11. Ahmed, W.A.M., Saad, E.S.M., Aziz, E.S.: Modified Back Propagation Algorithm for Learning Artificial Neural Networks. In: Proceedings of the Eighteenth National Radio Science Conference, pp. 345–352 (2001)

    Google Scholar 

  12. Nawi, N.M., et al.: Countering the Problem of Oscillations in Bat-BP Gradient Trajectory by Using Momentum. In: Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng 2013), pp. 103–110. Springer Singapore (2014)

    Google Scholar 

  13. Jin, W., et al.: The improvements of BP neural network learning algorithm. In: Proceedings of 5th International Conference on Signal Processing WCCC-ICSP 2000, vol. 3, pp. 1647–1649 (2000)

    Google Scholar 

  14. Nawi, N.M., Ransing, R.S., Salleh, M.N.M., Ghazali, R., Hamid, N.A.: An Improved Back Propagation Neural Network Algorithm on Classification Problems. In: Zhang, Y., Cuzzocrea, A., Ma, J., Chung, K.-i., Arslan, T., Song, X. (eds.) DTA and BSBT 2010. CCIS, vol. 118, pp. 177–188. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  15. Mohd Nawi, N., et al.: BPGD-AG: A New Improvement Of Back-Propagation Neural Network Learning Algorithms With Adaptive Gain. Journal of Science and Technology 2 (2011)

    Google Scholar 

  16. Nawi, N.M., Ghazali, R., Salleh, M.N.M.: The Development of Improved Back-Propagation Neural Networks Algorithm for Predicting Patients with Heart Disease. In: Zhu, R., Zhang, Y., Liu, B., Liu, C. (eds.) ICICA 2010. LNCS, vol. 6377, pp. 317–324. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  17. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5, 989–993 (1994)

    Article  Google Scholar 

  18. Wilamowski, B., et al.: Neural network trainer with second order learning algorithms. In: 11th International Conference on Intelligent Engineering Systems, INES 2007, pp. 127–132 (2007)

    Google Scholar 

  19. Lourakis, M.I.: A brief description of the Levenberg-Marquardt algorithm implemented by levmar (2005)

    Google Scholar 

  20. Xue, Q., et al.: Improved LMBP algorithm in the analysis and application of simulation data. In: 2010 International Conference on Computer Application and System Modeling (ICCASM), vol. 6, pp. 545–547 (2010)

    Google Scholar 

  21. Yan, J., et al.: Levenberg-Marquardt algorithm applied to forecast the ice conditions in Ningmeng Reach of the Yellow River. In: Fifth International Conference on Natural Computation, ICNC 2009, pp. 184–188 (2009)

    Google Scholar 

  22. Ozturk, C., Karaboga, D.: Hybrid artificial bee colony algorithm for neural network training. In: 2011 IEEE Congress on Evolutionary Computation (CEC), pp. 84–88 (2011)

    Google Scholar 

  23. Nawi, A.K.N.M., Rehman, M.Z.: A New Levenberg-Marquardt based Back-propagation Algorithm trained with Cuckoo Search. In: ICEEI 2013 Procedia Technology 8C, pp. 18–24 (2013)

    Google Scholar 

  24. Bäck, T., Schwefel, H.-P.: An overview of evolutionary algorithms for parameter optimization. Evolutionary Computation 1, 1–23 (1993)

    Article  Google Scholar 

  25. Blum, C., Socha, K.: Training feed-forward neural networks with ant colony optimization: An application to pattern classification. In: Fifth International Conference on Hybrid Intelligent Systems, HIS 2005, p. 6 (2005)

    Google Scholar 

  26. Karaboga, D., Akay, B., Ozturk, C.: Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. In: Torra, V., Narukawa, Y., Yoshida, Y. (eds.) MDAI 2007. LNCS (LNAI), vol. 4617, pp. 318–329. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  27. Montana, D.J., Davis, L.: Training Feedforward Neural Networks Using Genetic Algorithms. In: IJCAI, pp. 762–767 (1989)

    Google Scholar 

  28. Nawi, N.M., Rehman, M.Z., Khan, A.: A New Bat Based Back-Propagation (BAT-BP) Algorithm. In: Swiątek, J., Grzech, A., Swiątek, P., Tomczak, J.M. (eds.) Advances in Systems Science. AISC, vol. 240, pp. 395–404. Springer, Heidelberg (2014)

    Google Scholar 

  29. Zhang, J.Z.T.L.J., Lyu, M.: A hybrid particle swarm optimization back propagation algorithm for neural network training. Applied Mathematics and Computation 185, 1026–1037 (2007)

    Article  MATH  Google Scholar 

  30. Yang, X.-S.: Nature-inspired metaheuristic algorithms. Luniver Press (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Parit Raja, Johor, Malaysia

    Nazri Mohd Nawi, Abdullah khan & M. Z. Rehman

  2. School of Information Technology, Deakin University, Waurn Ponds, Geelong, VIC, Australia

    Maslina Abdul Aziz & Jemal H. Abawajy

  3. University of Malaya, 50603, Kuala Lumpur, Malaysia

    Tutut Herawan

  4. AMCS Research Center, Yogyakarta, Indonesia

    Tutut Herawan

Authors
  1. Nazri Mohd Nawi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdullah khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Z. Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maslina Abdul Aziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tutut Herawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jemal H. Abawajy
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Artificial Intelligence, Faculty of Computer Science and Information Technology Building, University of Malaya, 50603, Kuala Lumpur, Malaysia

    Chu Kiong Loo

  2. Department of Electronics and Communication Engineering, College of Engineering, Universiti Tenaga Nasional, Jalan IKRAM-UNITEN, 43009, Kajang, Selangor, Malaysia

    Keem Siah Yap

  3. School of Engineering and Information Technology, Murdoch University, South St., 6150, Murdoch, Western Australia, Australia

    Kok Wai Wong

  4. Department of Electrical and Electronics Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, 120-749, Seoul, South Korea

    Andrew Teoh

  5. Department of Electrical and Electronic Engineering, Xi’an Jiaotong-Liverpool University, Ren’ai Road 111, SIP 215123, Suzhou, Jiangsu Province, China

    Kaizhu Huang

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Nawi, N.M., khan, A., Rehman, M.Z., Aziz, M.A., Herawan, T., Abawajy, J.H. (2014). An Accelerated Particle Swarm Optimization Based Levenberg Marquardt Back Propagation Algorithm. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds) Neural Information Processing. ICONIP 2014. Lecture Notes in Computer Science, vol 8835. Springer, Cham. https://doi.org/10.1007/978-3-319-12640-1_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12640-1_30

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12639-5

  • Online ISBN: 978-3-319-12640-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation