Skip to main content
Springer Nature Link
Log in

Parallel Approach to the Levenberg-Marquardt Learning Algorithm for Feedforward Neural Networks

  • Conference paper
Artificial Intelligence and Soft Computing (ICAISC 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9119))

Included in the following conference series:

  • 2077 Accesses

Abstract

A parallel architecture of the Levenberg-Marquardt algorithm for training a feedforward neural network is presented. The proposed solution is based on completely new parallel structures to effectively reduce high computational load of this algorithm. Detailed parallel neural network structures are explicitely discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bilski, J.: The UD RLS algorithm for training the feedforward neural networks. International Journal of Applied Mathematics and Computer Science 15(1), 101–109 (2005)

    Google Scholar 

  2. Bilski, J., Litwiński, S., Smoląg, J.: Parallel realisation of QR algorithm for neural networks learning. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 158–165. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  3. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent RTRN neural network learning. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 11–16. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  4. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent Elman neural network learning. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 19–25. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent multi layer perceptron learning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 12–20. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  6. Bilski, J., Smoląg, J.: Parallel approach to learning of the recurrent Jordan neural network. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 32–40. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  7. Bilski, J.: Parallel Structures for Feedforward and Dynamical Neural Networks (in Polish). AOW EXIT (2013)

    Google Scholar 

  8. Bilski, J., Smoląg, J., Galushkin, A.I.: The parallel approach to the conjugate gradient learning algorithm for the feedforward neural networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2014, Part I. LNCS (LNAI), vol. 8467, pp. 12–21. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  9. Bilski, J., Smoląg, J.: Parallel Architectures for Learning the RTRN and Elman Dynamic Neural Networks. IEEE Transactions on Parallel and Distributed Systems PP(99) (2014), doi:10.1109/TPDS.2014.2357019

    Google Scholar 

  10. Chu, J.L., Krzyźak, A.: The recognition of partially occluded objects with support vector machines, convolutional neural networks and deep belief networks. Journal of Artificial Intelligence and Soft Computing Research 4(1), 5–19 (2014)

    Article  Google Scholar 

  11. Cpałka, K., Rutkowski, L.: Flexible Takagi-Sugeno Fuzzy Systems. In: Proceedings of the Int. Joint Conference on Neural Networks, Montreal, pp. 1764–1769 (2005)

    Google Scholar 

  12. Cpałka, K., Łapa, K., Przybył, A., Zalasiński, M.: A new method for designing neuro-fuzzy systems for nonlinear modelling with interpretability aspects. Neurocomputing 135, 203–217 (2014)

    Article  Google Scholar 

  13. Cpalka, K., Rebrova, O., Nowicki, R., et al.: On design of flexible neuro-fuzzy systems for nonlinear modelling. International Journal of General Systems 42(6), Special Issue: SI, 706–720 (2013)

    Google Scholar 

  14. Fahlman, S.: Faster learning variations on backpropagation: An empirical study. In: Proceedings of Connectionist Models Summer School, Los Atos (1988)

    Google Scholar 

  15. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5(6), 989–993 (1994)

    Article  Google Scholar 

  16. Korytkowski, M., Nowicki, R., Rutkowski, L., Scherer, R.: AdaBoost Ensemble of DCOG Rough–Neuro–Fuzzy Systems. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part I. LNCS, vol. 6922, pp. 62–71. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  17. Korytkowski, M., Rutkowski, L., Scherer, R.: From ensemble of fuzzy classifiers to single fuzzy rule base classifier. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 265–272. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  18. Korytkowski, M., Scherer, R.: Negative Correlation Learning of Neuro-fuzzy System Ensembles. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part I. LNCS (LNAI), vol. 6113, pp. 114–119. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  19. Laskowski, L., Jelonkiewicz, J.: Self-Correcting Neural Network for stereo-matching problem solving. Fundamenta Informaticae 138, 1–26 (2015)

    Article  MathSciNet  Google Scholar 

  20. Łapa, K., Przybył, A., Cpałka, K.: A new approach to designing interpretable models of dynamic systems. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 523–534. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  21. Łapa, K., Zalasiński, M., Cpałka, K.: A new method for designing and complexity reduction of neuro-fuzzy systems for nonlinear modelling. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 329–344. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  22. Marqardt, D.: An algorithm for last-sqares estimation of nonlinear paeameters. J. Soc. Ind. Appl. Math., 431–441 (1963)

    Google Scholar 

  23. Patan, K., Patan, M.: Optimal training strategies for locally recurrent neural networks. Journal of Artificial Intelligence and Soft Computing Research 1(2), 103–114 (2011)

    Google Scholar 

  24. Riedmiller, M., Braun, H.: A direct method for faster backpropagation learning: The RPROP Algorithm. In: IEEE International Conference on Neural Networks, San Francisco (1993)

    Google Scholar 

  25. Romaszewski, M., Gawron, P., Opozda, S.: Dimensionality reduction of dynamic msh animations using HO-SVD. Journal of Artificial Intelligence and Soft Computing Research 3(3), 277–289 (2013)

    Google Scholar 

  26. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McCelland, J. (red.) Parallel Distributed Processing, ch. 8, vol. 1. The MIT Press, Cambridge (1986)

    Google Scholar 

  27. Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)

    Article  Google Scholar 

  28. Rutkowski, L.: Identification of MISO nonlinear regressions in the presence of a wide class of disturbances. IEEE Transactions on Information Theory 37(1), 214–216 (1991)

    Article  MathSciNet  Google Scholar 

  29. Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: Decision trees for mining data streams based on the gaussian approximation. IEEE Transactions on Knowledge and Data Engineering 26(1), 108–119 (2014)

    Article  Google Scholar 

  30. Rutkowski, L., Przybył, A., Cpałka, K., Er, M.J.: Online speed profile generation for industrial machine tool based on neuro-fuzzy approach. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 645–650. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  31. Rutkowski, L., Rafajlowicz, E.: On optimal global rate of convergence of some nonparametric identification procedures. IEEE Transactions on Automatic Control 34(10), 1089–1091 (1989)

    Article  MathSciNet  Google Scholar 

  32. Smoląg, J., Bilski, J.: A systolic array for fast learning of neural networks. In: Proc. of V Conf. Neural Networks and Soft Computing, Zakopane, pp. 754–758 (2000)

    Google Scholar 

  33. Smoląg, J., Rutkowski, L., Bilski, J.: Systolic array for neural networks. In: Proc. of IV Conf. Neural Networks and Their Applications, Zakopane, pp. 487–497 (1999)

    Google Scholar 

  34. Starczewski, A.: A clustering method based on the modified RS validity index. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 242–250. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  35. Starczewski, J., Rutkowski, L.: Connectionist structures of type 2 Fuzzy Inference Systems. In: 4th International Conference on Parallel Processing and Applied Mathematics, Nalenczow, Poland (2001)

    Google Scholar 

  36. Starczewski, J., Rutkowski, L.: Interval type 2 neuro-fuzzy systems based on interval consequents. In: Neural Networks and Soft Computing. Advances In Soft Computing, pp. 570–577 (2003)

    Google Scholar 

  37. Tadeusiewicz, R.: Neural Networks (in Polish). AOW RM (1993)

    Google Scholar 

  38. Werbos, J.: Backpropagation through time: What it does and how to do it. Proceedings of the IEEE 78(10) (1990)

    Google Scholar 

  39. Wilamowski, B.M., Yo, H.: Neural network learning without backpropagation. IEEE Transactions on Neural Networks 21(11), 1793–1803 (2010)

    Article  Google Scholar 

  40. Zalasiński, M., Cpałka, K.: New approach for the on-line signature verification based on method of horizontal partitioning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part II. LNCS (LNAI), vol. 7895, pp. 342–350. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Czȩstochowa University of Technology, Czȩstochowa, Poland

    Jarosław Bilski & Jacek Smoląg

  2. Information Technology Institute, University of Sociel Sciences, Łódź, Poland

    Jacek M. Żurada

  3. Department Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40292, USA

    Jacek M. Żurada

Authors
  1. Jarosław Bilski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Smoląg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacek M. Żurada
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jarosław Bilski .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Częstochowa, Poland

    Rafal Scherer

  4. AGH University of Science and Technology, Krakow, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Bilski, J., Smoląg, J., Żurada, J.M. (2015). Parallel Approach to the Levenberg-Marquardt Learning Algorithm for Feedforward Neural Networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2015. Lecture Notes in Computer Science(), vol 9119. Springer, Cham. https://doi.org/10.1007/978-3-319-19324-3_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19324-3_1

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19323-6

  • Online ISBN: 978-3-319-19324-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics