Skip to main content
SpringerLink
Log in

The Parallel Approach to the Conjugate Gradient Learning Algorithm for the Feedforward Neural Networks

  • Conference paper
Artificial Intelligence and Soft Computing (ICAISC 2014)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 8467))

Included in the following conference series:

Abstract

This paper presents the parallel architecture of the conjugate gradient learning algorithm for the feedforward neural networks. The proposed solution is based on the high parallel structures to speed up learning performance. Detailed parallel neural network structures are explicitly shown.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bilski, J.: The UD RLS algorithm for training the feedforward neural networks. International Journal of Applied Mathematics and Computer Science 15(1), 101–109 (2005)

    Google Scholar 

  2. Bilski, J., Litwiński, S., Smoląg, J.: Parallel realisation of QR algorithm for neural networks learning. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 158–165. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  3. Bilski, J., Smoląg, J.: Parallel realisation of the RTRN neural network learning. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 11–16. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  4. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent Elman neural network learning. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 19–25. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Bilski, J., Smoląg, J.: Parallel realisation of the recurrent multi layer perceptron learning. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS (LNAI), vol. 7267, pp. 12–20. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  6. Bilski, J., Smoląg, J.: Parallel approach to learning of the recurrent Jordan neural network. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2013, Part I. LNCS (LNAI), vol. 7894, pp. 32–40. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  7. Charalambous, C.: Conjugate gradient algorithm for efficient training of artificial neural networks. IEE Proc.-G 139(3), 301–310 (1992)

    Article  Google Scholar 

  8. Fahlman, S.: Faster learning variations on backpropagation: An empirical study. In: Proceedings of Connectionist Models Summer School, Los Atos (1988)

    Google Scholar 

  9. Fletcher, R., Powell, M.J.D.: A rapidly convergent descent method for minimization. Computer Journal 6, 163–168 (1963)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Computer Journal 7, 149–154 (1964)

    Article  MATH  MathSciNet  Google Scholar 

  11. Korytkowski, M., Rutkowski, L., Scherer, R.: On combining backpropagation with boosting. In: IEEE International Joint Conference on Neural Network (IJCNN) Proceedings, Vancouver, July 16-21, vols. 1-10, pp. 1274–1277 (2006)

    Google Scholar 

  12. Hagan, M.T., Menhaj, M.B.: Training feedforward networks with the Marquardt algorithm. IEEE Transactions on Neural Networks 5(6) (1994)

    Google Scholar 

  13. Li, X., Er, M.J., Lim, B.S., et al.: Fuzzy regression modelling for tool performance prediction and degradation detection. International Journal of Neural Systems 20(5), 405–419 (2010)

    Article  Google Scholar 

  14. Nocedal, J., Wright, S.J.: Conjugate Gradient Methods in Numerical Optimization, pp. 497–528. Springer, New York (2006)

    Google Scholar 

  15. Polak, E.: Computational methods in optimization: a unified approach. Academic Press, New York (1971)

    Google Scholar 

  16. Riedmiller, M., Braun, H.: A direct method for faster backpropagation learning: The RPROP Algorithm. In: IEEE International Conference on Neural Networks, San Francisco (1993)

    Google Scholar 

  17. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McCelland, J. (ed.) Parallel Distributed Processing, vol. 1, ch. 8. The MIT Press, Cambridge (1986)

    Google Scholar 

  18. Rutkowski, L.: Multiple Fourier series procedures for extraction of nonlinear regressions from noisy data. IEEE Transactions on Signal Processing 41(10), 3062–3065 (1993)

    Article  MATH  Google Scholar 

  19. Rutkowski, L.: Non-parametric learning algorithms in the time-varying environments. Signal Processing 18(2), 129–137 (1989)

    Article  MathSciNet  Google Scholar 

  20. Rutkowski, L.: Generalized regression neural networks in time-varying environment. IEEE Trans. Neural Networks 15, 576–596 (2004)

    Article  Google Scholar 

  21. Rutkowski, L., Cpałka, K.: Compromise approach to neuro-fuzzy systems. In: Sincak, P., Vascak, J., Kvasnicka, V., Pospichal, J. (eds.) Intelligent Technologies - Theory and Applications, vol. 76, pp. 85–90. IOS Press (2002)

    Google Scholar 

  22. Rutkowski, L., Cpałka, K.: Neuro-fuzzy systems derived from quasi-triangular norms. In: Proceedings of the IEEE International Conference on Fuzzy Systems, Budapest, July 26-29, vol. 2, pp. 1031–1036 (2004)

    Google Scholar 

  23. Rutkowski, L., Przybył, A., Cpałka, K.: Novel Online Speed Profile Generation for industrial machine tool based on flexible neuro-fuzzy approximation. IEEE Transactions on Industrial Electronics 59(2), 1238–1247 (2012)

    Article  Google Scholar 

  24. Smoląg, J., Bilski, J.: A systolic array for fast learning of neural networks. In: Proc. of V Conf. Neural Networks and Soft Computing, Zakopane, pp. 754–758 (2000)

    Google Scholar 

  25. Smoląg, J., Rutkowski, L., Bilski, J.: Systolic array for neural networks. In: Proc. of IV Conf. Neural Networks and Their Applications, Zakopane, pp. 487–497 (1999)

    Google Scholar 

  26. Starczewski, J.T.: A type-1 approximation of interval type-2 FLS. In: Di Gesù, V., Pal, S.K., Petrosino, A. (eds.) WILF 2009. LNCS, vol. 5571, pp. 287–294. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  27. Tadeusiewicz, R.: Neural Networks, AOW RM (1993) (in Polish)

    Google Scholar 

  28. Werbos, J.: Backpropagation through time: What it does and how to do it. Proceedings of the IEEE 78(10) (1990)

    Google Scholar 

  29. Wilamowski, B.M., Yo, H.: Neural network learning without backpropagation. IEEE Transactions on Neural Networks 21(11), 1793–1803 (2010)

    Article  Google Scholar 

  30. Żurada, J.: Introduction to Artificial Neural Systems. West Publishing Company (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Intelligence, Częstochowa University of Technology, Częstochowa, Poland

    Jarosław Bilski & Jacek Smoląg

  2. Moscow Institute of Physics and Technology, Russia

    Alexander I. Galushkin

Authors
  1. Jarosław Bilski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Smoląg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander I. Galushkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Armii Krajowej 36, 42-200, Częstochowa, Poland

    Leszek Rutkowski , Marcin Korytkowski  & Rafał Scherer ,  & 

  2. AGH University of Science and Technology, Mickiewicza 30, 30-059, Kraków, Poland

    Ryszard Tadeusiewicz

  3. Computer Science Division, Department of Electrical Engineering and Computer Sciences, University of California Berkeley, 94720-1776, Berkeley, CA, USA

    Lotfi A. Zadeh

  4. Computational Intelligence Laboratory, Electrical and Computer Engineering, University of Louisville, 405 Lutz Hall, 40292, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Bilski, J., Smoląg, J., Galushkin, A.I. (2014). The Parallel Approach to the Conjugate Gradient Learning Algorithm for the Feedforward Neural Networks. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2014. Lecture Notes in Computer Science(), vol 8467. Springer, Cham. https://doi.org/10.1007/978-3-319-07173-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-07173-2_2

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-07172-5

  • Online ISBN: 978-3-319-07173-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation