Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Neural Networks

ISNN 2017: Advances in Neural Networks - ISNN 2017 pp 547–554Cite as

A Caputo-Type Fractional-Order Gradient Descent Learning of BP Neural Networks

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10261))

Abstract

Fractional calculus has been found to be a promising area of research for information processing and modeling of some physical systems. In this paper, we propose a fractional gradient descent method for the backpropagation (BP) training of neural networks. In particular, the Caputo derivative is employed to evaluate the fractional-order gradient of the error defined as the traditional quadratic energy function. Simulation has been implemented to illustrate the performance of presented fractional-order BP algorithm on large dataset: MNIST.

J. Wang—This work was supported in part by the National Natural Science Foundation of China (No. 61305075), the China Postdoctoral Science Foundation (No. 2012M520624), the Natural Science Foundation of Shandong Province (Nos. ZR2013FQ004, ZR2013DM015), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20130133120014) and the Fundamental Research Funds for the Central Universities (Nos. 14CX05042A, 15CX05053A, 15CX02079A, 15CX08011A).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Chen, X.: Application of fractional calculus in BP neural networks (in Chinese). Master thesis. Nanjing Forestry University, Nanjing, Jiangsu (2013)

    Google Scholar 

  2. Zhang, S., Yu, Y., Wang, H.: Mittag-Leffler stability of fractional-order Hopfield neural networks. Nonlinear Anal. Hybri. 16, 104–121 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  3. Chen, B., Chen, J.: Global \(O(t^{-\alpha })\) stability and global asymptotical periodicity for a non-autonomous fractional-order neural networks with time varying delays. Neural Netw. 73, 47–57 (2016)

    Article  Google Scholar 

  4. Rakkiyappan, R., Sivaranjani, R., Velmurugan, G., Cao, J.: Analysis of global \(O(t^{-\alpha })\) stability and global asymptotical periodicity for a class of fractional-order complex-valued neural networks with time varying dela. Neural Netw. 77, 51–69 (2016)

    Article  Google Scholar 

  5. Rakkiyappan, R., Cao, J., Velmurugan, G.: Existence and uniform stability analysis of fractional-order complex-valued neural networks with time delays. IEEE Trans. Neural Netw. Learn. 26, 84–97 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Xiao, M., Zheng, W., Jiang, G., Cao, J.: Undamped oscillations generated by Hopf Bifurcations in fractional-order recurrent neural networks with Caputo derivative. IEEE Trans. Neural Netw. Learn. 26, 3201–3214 (2015)

    Article  MathSciNet  Google Scholar 

  7. Wang, H., Yu, Y., Wen, G.: Stability analysis of fractional-order Hopfield neural networks with time delays. Neural Netw. 55, 98–109 (2014)

    Article  MATH  Google Scholar 

  8. Wu, A., Zhang, J., Zen, Z.: Dynamic behaviors of a class of memristor-based Hopfield networks. Phys. Lett. A. 375, 1661–1665 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  9. Wu, A., Wen, S., Zen, Z.: Anti-synchronization control of a class of memristive recurrent neural networks. Commun. Nonlinear Sci. 18, 373–385 (2013)

    Article  MathSciNet  Google Scholar 

  10. Pu, Y., Zhou, J., Zhang, Y., Zhang, N., Huang, G., Siarry, P.: Fractional extreme value adaptive training method: fractional steepest descent approach. IEEE Trans. Neural Netw. Learn. 26, 653–662 (2015)

    Article  MathSciNet  Google Scholar 

  11. Love, E.R.: Fractional derivatives of imaginary order. J. Lond. Math. Soc. 3, 241–259 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  12. Oldham, K.B., Spanier, J.: The Fractional Calculus: Theory and Applications of Differentiation and Integration to Arbitrary Order. Academic, Cambridge (1974)

    MATH  Google Scholar 

  13. Mcbride, A.C.: Fractional Calculus. Halsted, USA (1986)

    Google Scholar 

  14. Nishimoto, K.: Fractional Calculus: Integrations and Differentiations of Arbitrary Order. New Haven University Press, New Haven (1989)

    MATH  Google Scholar 

  15. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Science, China University of Petroleum, Qingdao, 266580, China

    Guoling Yang, Bingjie Zhang, Zhaoyang Sang, Jian Wang & Hua Chen

Authors
  1. Guoling Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaoyang Sang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hua Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Wang .

Editor information

Editors and Affiliations

  1. Dalian University of Technology, Dalian, China

    Fengyu Cong

  2. City University of Hong Kong, Kowloon Tong, Hong Kong

    Andrew Leung

  3. Chinese Academy of Sciences, Beijing, China

    Qinglai Wei

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Yang, G., Zhang, B., Sang, Z., Wang, J., Chen, H. (2017). A Caputo-Type Fractional-Order Gradient Descent Learning of BP Neural Networks. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10261. Springer, Cham. https://doi.org/10.1007/978-3-319-59072-1_64

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59072-1_64

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59071-4

  • Online ISBN: 978-3-319-59072-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation