Skip to main content
SpringerLink
Log in

Conjugate Gradient Algorithms for Complex-Valued Neural Networks

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9490))

Included in the following conference series:

Abstract

In this paper, conjugate gradient algorithms for complex-valued feedforward neural networks are proposed. Since these algorithms yielded better training results for the real-valued case, an extension to the complex-valued case is a natural option to enhance the performance of the complex backpropagation algorithm. The full deduction of the classical variants of the conjugate gradient algorithm is presented, and the resulting training methods are exemplified on synthetic and real-world applications. The experimental results show a significant improvement over the complex gradient descent algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amin, M., Savitha, R., Amin, M., Murase, K.: Complex-valued functional link network design by orthogonal least squares method for function approximation problems. In: International Joint Conference on Neural Networks (IJCNN), pp. 1489–1496. IEEE, July 2011

    Google Scholar 

  2. Arena, P., Fortuna, L., Re, R., Xibilia, M.: Multilayer perceptrons to approximate complex valued functions. Int. J. Neural Syst. 6(4), 435–446 (1995)

    Article  Google Scholar 

  3. Beale, E.: A derivation of conjugate gradients. In: Lootsma, F.A. (ed.) Numerical Methods for Nonlinear Optimization, pp. 39–43. Academic Press, London (1972)

    Google Scholar 

  4. Bishop, C.: Neural Networks for Pattern Recognition. Oxford University Press Inc., New York (1995)

    MATH  Google Scholar 

  5. Buchholz, S., Sommer, G.: On clifford neurons and clifford multi-layer perceptrons. Neural Netw. 21(7), 925–935 (2008)

    Article  MATH  Google Scholar 

  6. Charalambous, C.: Conjugate gradient algorithm for efficient training of artificial neural networks. IEEE Proc. G Circ. Devices Syst. 139(3), 301–310 (1992)

    Article  Google Scholar 

  7. Dai, Y., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  8. Gilbert, J., Nocedal, J.: Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2(1), 21–42 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  9. Goh, S., Mandic, D.: A complex-valued rtrl algorithm for recurrent neural networks. Neural Comput. 16(12), 2699–2713 (2004)

    Article  MATH  Google Scholar 

  10. Goh, S., Mandic, D.: Nonlinear adaptive prediction of complex-valued signals by complex-valued prnn. IEEE Trans. Signal Process. 53(5), 1827–1836 (2005)

    Article  MathSciNet  Google Scholar 

  11. Goh, S., Mandic, D.: Stochastic gradient-adaptive complex-valued nonlinear neural adaptive filters with a gradient-adaptive step size. IEEE Trans. Neural Netw. 18(5), 1511–1516 (2007)

    Article  Google Scholar 

  12. Hestenes, M., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl Bur. Stan. 49(6), 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hirose, A.: Complex-Valued Neural Networks: Advances and Applications. Wiley, New York (2013)

    Book  Google Scholar 

  14. Huang, G.B., Li, M.B., Chen, L., Siew, C.K.: Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71(4–6), 576–583 (2008)

    Article  Google Scholar 

  15. Johansson, E., Dowla, F., Goodman, D.: Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method. Int. J. Neural Syst. 2(4), 291–301 (1991)

    Article  Google Scholar 

  16. Luenberger, D., Ye, Y.: Linear and Nonlinear Programming. International Series in Operations Research & Management Science, vol. 116. Springer, US (2008)

    MATH  Google Scholar 

  17. Narendra, K., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1(1), 4–27 (1990)

    Article  Google Scholar 

  18. Polak, E., Ribiere, G.: Note sur la convergence de méthodes de directions conjuguées. Rev. Fr. d’Informatique et de Rech. Opérationnelle 3(16), 35–43 (1969)

    MathSciNet  MATH  Google Scholar 

  19. Powell, M.: Restart procedures for the conjugate gradient method. Math. Program. 12(1), 241–254 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  20. Reeves, C., Fletcher, R.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  21. Savitha, R., Suresh, S., Sundararajan, N.: A fully complex-valued radial basis function network and its learning algorithm. Int. J. Neural Syst. 19(4), 253–267 (2009)

    Article  Google Scholar 

  22. Savitha, R., Suresh, S., Sundararajan, N.: A self-regulated learning in fully complex-valued radial basis function networks. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE, July 2010

    Google Scholar 

  23. Savitha, R., Suresh, S., Sundararajan, N.: A meta-cognitive learning algorithm for a fully complex-valued relaxation network. Neural Netw. 32, 209–218 (2012)

    Article  MATH  Google Scholar 

  24. Widrow, B., McCool, J., Ball, M.: The complex lms algorithm. Proc. IEEE 63(4), 719–720 (1975)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Software Engineering, Polytechnic University Timişoara, Blvd. V. Pârvan, No. 2, 300223, Timişoara, Romania

    Călin-Adrian Popa

Authors
  1. Călin-Adrian Popa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Călin-Adrian Popa .

Editor information

Editors and Affiliations

  1. University of Istanbul, Istanbul, Turkey

    Sabri Arik

  2. University at Qatar, Doha, Qatar

    Tingwen Huang

  3. Tunku Abdul Rahman University College, Kuala Lumpur, Malaysia

    Weng Kin Lai

  4. University of Science Technology, Wuhan, China

    Qingshan Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Popa, CA. (2015). Conjugate Gradient Algorithms for Complex-Valued Neural Networks. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9490. Springer, Cham. https://doi.org/10.1007/978-3-319-26535-3_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26535-3_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26534-6

  • Online ISBN: 978-3-319-26535-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation