Skip to main content
Springer Nature Link
Log in

Statistical Neurodynamics for Sequence Processing Neural Networks with Finite Dilution

  • Conference paper
Advances in Neural Networks – ISNN 2007 (ISNN 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4491))

Included in the following conference series:

  • 1974 Accesses

Abstract

We extend the statistical neurodynamics to study transient dynamics of sequence processing neural networks with finite dilution, and the theoretical results are supported by extensive numerical simulations. It is found that the order parameter equations are completely equivalent to those of the Generating Functional Method, which means that crosstalk noise follows normal distribution even in the case of failure in retrieval process. In order to verify the gaussian assumption of crosstalk noise, we numerically obtain the cumulants of crosstalk noise, and third- and fourth-order cumulants are found to be indeed zero even in non-retrieval case.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Sompolinsky, H., Kanter, I.: Temporal Association in Aymmetric Neural Networks. Phys. Rev. Lett. 57, 2861–2864 (1986)

    Article  Google Scholar 

  2. Düring, A., Coolen, A.C.C., Sherrington, D.: Phase Diagram and Storage Capacity of Sequence Processing Neural Networks. J. Phys. A: Math. Gen. 31, 8607–8621 (1998)

    Article  MATH  Google Scholar 

  3. Kawamura, M., Okada, M.: Transient Dynamics for Sequence Processing Neural Networks. J. Phys. A: Math. Gen. 35, 253–266 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  4. Theumann, W.K.: Mean-field Dynamics of Sequence Processing Neural Networks with Finite Connectivity. Physica A 328, 1–12 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  5. Yong, C., Hai, W.Y., Qing, Y.K.: The Attractors in Sequence Processing Neural Networks. Int. J. Modern Phys. C 11, 33–39 (2000)

    Article  Google Scholar 

  6. Hopfield, J.J.: Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Nat. Acad. Sci. 79, 2554–2558 (1982)

    Article  MathSciNet  Google Scholar 

  7. Amit, D.J., Gutfreund, H., Sompolinsky, H.: Spin-glass Models of Neural Networks. Phys. Rev. A. 32, 1007–1018 (1985)

    Article  MathSciNet  Google Scholar 

  8. Amari, S.: Statistical Neurodynamics of Associative Memory. In: Proc. IEEE Conference on Neural Networks., vol. 1, pp. 633–640 (1988)

    Google Scholar 

  9. Okada, M.: A Hierarchy of Macrodynamical Equations for Associative Memory. Neural Networks 8, 833–838 (1995)

    Article  Google Scholar 

  10. Nishimori, H., Ozeki, T.: Retrieval Dynamics of Associative Memory of the Hopfield Type. J. Phys. A: Math. Gen. 26, 859–871 (1993)

    Article  MATH  Google Scholar 

  11. Ozeki, T., Nishimori, H.: Noise Distributions in Retrieval Dynamics of the Hopfield Model. J. Phys. A: Math. Gen. 27, 7061–7068 (1994)

    Article  MATH  Google Scholar 

  12. Kitano, K., Aoyagi, T.: Retrieval Dynamics of Neural Networks for Sparsely Coded Sequential Patterns. J. Phys. A: Math. Gen. 31, L613–L620 (1998)

    Google Scholar 

  13. Gardner, E., Derrida, B., Mottishaw, P.: Zero Temperature Parallel Dynamics for Infinite Range Spin Glasses and Neural Networks. J. Physique 48, 741–755 (1987)

    Article  Google Scholar 

  14. Sommers, H.J.: Path-integral Approach to Ising Spin-glass Dynamics. Phys. Rev. Lett. 58, 1268–1271 (1987)

    Article  Google Scholar 

  15. Gomi, S., Yonezawa, F.: A New Perturbation Theory for the Dynamics of the Little-Hopfield Model. J. Phys. A: Math. Gen. 28, 4761–4775 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  16. Koyama, H., Fujie, N., Seyama, H.: Results From the Gardner-Derrida-Mottishaw Theory of Associative Memory. Neural Networks 12, 247–257 (1999)

    Article  Google Scholar 

  17. Coolen, A.C.C.: Statistical Mechanics of Recurrent Neural Networks II. Dynamics. cond-mat/0006011

    Google Scholar 

  18. Watkin, T.L.H., Sherrington, D.: The Parallel Dynamics of a Dilute Symmetric Hebb-rule Network. J. Phys A: Math. Gen. 24, 5427–5433 (1991)

    Article  MathSciNet  Google Scholar 

  19. Derrida, B., Gardner, E., Zippelius, A.: An Exactly Solvable Asymmetric Neural Network Model. Europhys. Lett. 4, 167–173 (1987)

    Article  Google Scholar 

  20. Patrick, A.E., Zagrebnov, V.A.: Parallel Dynamics for an Extremely Diluted Neural Network. J. Phys. A: Math. Gen. 23, L1323–L1329 (1990)

    Google Scholar 

  21. Castillo, I.P., Skantzos, N.S.: The Little-Hopfield Model on a Random Graph. cond-mat/0307499

    Google Scholar 

  22. Sompolinsky, H.: Neural Networks with Nonlinear Synapses and Static Noise. Phys. Rev. A. 34, 2571–2574 (1986)

    Article  Google Scholar 

  23. Wemmenhove, B., Coolen, A.C.C.: Finite Connectivity Attractor Neural Networks. J. Phys A: Math. Gen. 36, 9617–9633 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  24. Chen, Y., Wang, Y.H., Yang, K.Q.: Macroscopic Dynamics in Separable Neural Networks. Phys. Rev. E 63, 041901-4 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Theoretical Physics, Lanzhou University, 730000 Lanzhou, China

    Pan Zhang & Yong Chen

Authors
  1. Pan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering (M/C 154), University of Illinois at Chicago, 851 S. Morgan Street, 60607-7053, Chicago, IL, USA

    Derong Liu

  2. School of Automation, Southeast University, 210096, Nanjing, China

    Shumin Fei

  3. Laboratory of Complex Systems, Institute of Automation, Chinese Adacemy of Sciences, 100080, Beijing, P. R. China

    Zeng-Guang Hou

  4. School of Information Science and Engineering, Northeast University, Shenyang, 110004, China

    Huaguang Zhang

  5. School of Electrical Engineering, Hohai University, Nanjing, 210098, China

    Changyin Sun

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhang, P., Chen, Y. (2007). Statistical Neurodynamics for Sequence Processing Neural Networks with Finite Dilution. In: Liu, D., Fei, S., Hou, ZG., Zhang, H., Sun, C. (eds) Advances in Neural Networks – ISNN 2007. ISNN 2007. Lecture Notes in Computer Science, vol 4491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72383-7_134

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72383-7_134

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72382-0

  • Online ISBN: 978-3-540-72383-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics