Skip to main content
SpringerLink
Log in

Feed-back neural networks with discrete weights

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

We use the Monte Carlo Adaptation learning algorithm to design feed-back neural networks with discrete weights. The dynamic properties of these types of neural networks are investigated as a function of the states of weights. The numerical results of these networks show three phases: the “chaos phase,” the “pure memory phase” and the “mixture phase” in the parameter space. The maximum storage ratio for the “pure memory phase” increases with the increasing of the states of the weights, which is favorable for practical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Amaldi E (1991) On the complexity of training perceptrons. In: International conference on artificial neural networks. Helsinki Published by Elsvier, pp 55–60

  2. Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann Phys 173:30

    Article  Google Scholar 

  3. Amit DJ (1989) Modeling brain functions. Cambridge University Press, Cambridge

    Google Scholar 

  4. Amit DJ, Fusi S (1992) Constraints on learning in dynamic synapses. Network 3:443

    Article  MATH  Google Scholar 

  5. Athithan G, Dasgupta C (1997) On the problem of spurious patterns in neural associative memory models. IEEE Trans Neural Netw 8:1483–1491

    Article  Google Scholar 

  6. Braunstein A, Mezard M, Zecchina R (2005) Survey propagation: an algorithm for satisfiability. Random Struct Algorithms 27:201–226

    Article  MathSciNet  MATH  Google Scholar 

  7. Braunstein A, Zecchina R (2006) Learning by message-passing in networks of discrete synapses. Phys Rev Lett 96:030201

    Article  MathSciNet  Google Scholar 

  8. Blum AL, Rivest RL (1992) Training a 3-node neural network is NP-complete. Neural Netw 5:117–127

    Article  Google Scholar 

  9. Baldassi C, Braunstein A, Brunel N, Zecchina R (2007) Efficient supervised learning in networks with binary synapses. Proc Natl Acad Sci USA 104:11079

    Article  Google Scholar 

  10. Gardner E (1988) The space of interactions in neural networks. J Phy A 21:257–270

    Article  Google Scholar 

  11. Gutfreund H, Stein Y (1990) Capacity of neural networks with discrete synaptic couplings. J Phys A Math Gen 23:2613–2630

    Article  MathSciNet  MATH  Google Scholar 

  12. Horner H (1992) Dynamics of learning for the binary perceptron problem. Z Phys B B86:291

    Article  Google Scholar 

  13. Hopfield JJ, Feinstein DI, Palmer RG (1983) Unlearning has a stabilizing effect in collective memories. Nature 304:158–159

    Article  Google Scholar 

  14. Jin T, Zhao H (2005) Pattern recognition using asymmetric attractor neural networks. Phys Rev E 72:1539–3755

    Google Scholar 

  15. Krauth W, Nadal JP, Mezard M (1988) The roles of stability and symmetry in the dynamics of neural networks. J Phys A Math Gen 21:2995–3011

    Article  MathSciNet  MATH  Google Scholar 

  16. Krauth W, Mzard M (1989) Storage capacity of memory networks with binary couplings. J de Physique 50:3057

    Article  Google Scholar 

  17. Lehmann N, Sommers HJ (1991) Eigenvalue statistics of random real matrices. Phys Rev Lett 67:941–944

    Article  MathSciNet  MATH  Google Scholar 

  18. Mezard M, Parisi G, Zecchina R (2002) Analysis and algorithmic solutions of random satisfiability problems. Science 297:812–815

    Article  Google Scholar 

  19. O’Connor DH, Wittenberg GM, Wang SSH (2005) Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc Natl Acad Sci USA 102:9679–9684

    Article  Google Scholar 

  20. Opper M, Krauth W (1989) Critical storage capacity of the j =  +1 neural network. J Phy A Math Gen 22:L519–L523

    Article  Google Scholar 

  21. Petersen CCH, Malenka RC, Nicoll RA, Hopfield JJ (1998) All-or-none potentiation at CA3-CA1 synapses. Proc Natl Acad Sci USA 95:4732–4737

    Article  Google Scholar 

  22. Schuster HG (2001) Complex adaptive systems. Scator Verlag, Saarbrucken

    Google Scholar 

  23. Sommers HJ, Crisanti A, Sompolinsky H, Stein Y (1988) Spectrum of large random asymmetric matrices. Phys Rev Lett 60:1895–1898

    Article  MathSciNet  Google Scholar 

  24. Tomlinson MS Jr (1988) Discrete weight neural network. United States Patent

  25. Tsodyks MV (1990) Associative memory in neural networks with binary synapses. Mod Phys Lett B B4:713–716

    Article  MathSciNet  Google Scholar 

  26. Turel O, Lee JH, Ma X, Likharev KK (2004) Neuromorphic architectures for nanoelectronic circuits. Int J Circ Theor Appl 32:277–302

    Article  Google Scholar 

  27. Zhao H (2004) Designing asymmetric neural networks with associative memory. Phys Rev E 70:066137

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank Professor Nicholas McGuire for useful discussions and suggestions. Thanks, also to Schuster for useful discussions and for introducing us to the literature and references [17, 23] for this study. Finally, we want to thank Dr. Zhang Yong for their constructive discussion. This work was supported by National Natural Science Foundation of China under Grant No. 60973137, No. 10775115, No. 10975115 and No. 10925525; Gansu Sci. & Tech. Support Program under Grant No. 1104GKCA049; The Fundamental Research Funds for the Central Universities under Grant No. lzujbky-2010-89 and Google Faculty Award.

Author information

Authors and Affiliations

  1. School of Information Science and Engineer, Lanzhou University, Lanzhou, 730000, China

    Lijuan Wang & Qingguo Zhou

  2. Faculty of Informatics, University of Wollongong, Wollongong, NSW, Australia

    Lijuan Wang

  3. Department of Physics, Xiamen University, Xiamen, 361005, China

    Tao Jin & Hong Zhao

Authors
  1. Lijuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingguo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingguo Zhou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, L., Zhou, Q., Jin, T. et al. Feed-back neural networks with discrete weights. Neural Comput & Applic 22, 1063–1069 (2013). https://doi.org/10.1007/s00521-012-0867-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-0867-8

Keywords

Search

Navigation