Skip to main content

Advertisement

Springer Nature Link
Log in

Finite Memory Loading in Hairy Neurons

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

This paper presents a method to expand the basins of stable patterns in associative memory. It examines fully-connected associative memory geometrically and translate the learning process into an algebraic optimization procedure. It finds that locating all the patterns at certain stable corners of the neurons’ hypercube as far from the decision hyperplanes as possible can produce excellent error tolerance. It then devises a method based on this finding to develop the hyperplanes. This paper further shows that this method leads to the hairy model, or the deterministic analogue of the Gibb’s free energy model. Through simulations, it shows that this method gives better error tolerance than does the Hopfield model and the error-correction rule in both synchronous and asynchronous modes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Abbreviations

AM:

associative memory

EAM:

expanded associative memory

ECR:

error-correction rule

LM:

Little model

RK:

Runge-Kutta method

References

  • DH Ackley GE Hinton TJ. Sejnowski (1985) ArticleTitleA learning algorithm for Boltzmann machine Cognitive Science. 9 147–169 Occurrence Handle10.1016/S0364-0213(85)80012-4

    Article  Google Scholar 

  • SI. Amari (1972) ArticleTitleLearning patterns and pattern sequences by self-organising nets IEEE Transactions on Computers. 21 1197–1206 Occurrence Handle0245.94024 Occurrence Handle51 #2373

    MATH  MathSciNet  Google Scholar 

  • Boser B, Guyon I, Vapnik VN. (1992). A training algorithm for optimal margin classifiers. In: Fifth Annual Workshop on Computational Learning Theory, Morgan Kaufmann, pp. 144–152

  • J. Bruck (1990) ArticleTitleOn the convergence properties of the Hopfield model Proceeding of IEEE. 78 1579–1585

    Google Scholar 

  • TM. Cover (1965) ArticleTitleGeometrical and statistical properties of systems of linear inequalities with applications in pattern recognition IEEE Transactions on Electronic Computers. 14 326–334 Occurrence Handle0152.18206

    MATH  Google Scholar 

  • E. Gardner (1987) ArticleTitleMaximum storage capacity in neural networks Electrophysics Letters. 4 IssueID4 481–485

    Google Scholar 

  • E. Gardner (1989) ArticleTitleOptimal basins of attraction in randomly sparse neural network models Journal of Physics. A 22 IssueID12 1969–1974 Occurrence Handle90f:82028

    MathSciNet  Google Scholar 

  • DO. Hebb (1949) The Organization of Behavior: A Neuropsychological Theory Wiley New York

    Google Scholar 

  • JJ. Hopfield (1982) ArticleTitleNeural networks and physical systems with emergent collective computational ability Proceeding of the National Academy of Science. 79 2554–2558 Occurrence Handle83g:92024

    MathSciNet  Google Scholar 

  • Ince DC. (1992). Intelligent machinery. In: Ince DC. (eds). Collected Works of A. M. Turing: Mechanical Intelligence. Elsevier Science Publishers

  • I Kanter H. Sompolinsky (1987) ArticleTitleAssociative recall of memory without errors Physics Review. A 35 IssueID1 380–392

    Google Scholar 

  • Kauffman SA (1991) Antichaos and adaptation. Scientific American: 64–70

  • J Li AN Michel W. Porod (1989) ArticleTitleAnalysis and synthesis of a class of neural networks: linear systems operating on a closed hypercube IEEE Transactions on Circuits and Systems. 36 IssueID11 1405–1422 Occurrence Handle10.1109/31.41297 Occurrence Handle90k:92015

    Article  MathSciNet  Google Scholar 

  • Liou CY and Lin SL (1989) The other variant Boltzmann machine. In: Proceedings of the IJCNN, Washington DC, pp. 449–454

  • Liou CY and Sou UC. (2003) Loading temporal associative memory using the neuronic equation. In: Kaynak O, Alpaydin E, Oja E and Xu L (eds), LCS, vol. 2714, Springer, pp. 52–59

  • CY Liou JM. Wu (1996) ArticleTitleSelf-organization using Potts models Neural Networks. 9 IssueID4 671–684 Occurrence Handle10.1016/0893-6080(95)00111-5

    Article  Google Scholar 

  • Liou CY and Yu WJ (1995) Ambiguous binary representation in multilayer neural network. In: Proceedings of the ICNN, Perth, Australia, vol. 1, pp. 379–384

  • CY Liou SK. Yuan (1999) ArticleTitleError tolerant associative memory Biological Cybernetics. 81 331–342 Occurrence Handle10.1007/s004220050566

    Article  Google Scholar 

  • WA. Little (1974) ArticleTitleThe existence of persistent states in the brain Mathematical Biosciences. 19 101–120 Occurrence Handle10.1016/0025-5564(74)90031-5 Occurrence Handle0272.92011

    Article  MATH  Google Scholar 

  • RJ Mceliece EC Posner ER Rodemich SS. Venkatesh (1987) ArticleTitleThe capacity of the Hopfield associative memory IEEE Transaction on information Theory. 33 461–482 Occurrence Handle88h:92019

    MathSciNet  Google Scholar 

  • Szu H. (1989). Reconfigurable neural nets by energy convergence learning principle based on extended McCulloch–Pitts neurons and synapses. In: Proc IJCNN, Washington, DC, vol. 1, pp. 485–496

  • H. Szu (1999) ArticleTitleThermodynamics energy for both supervised and unsupervised learning neural nets at a constant temperature International Journal of Neural System. 9 175–186

    Google Scholar 

  • Q Tao T Fang H. Qiao (2001) ArticleTitleA novel continuous-time neural network for realizing associative memory IEEE Transactions on Neural Networks. 12 IssueID2 418–423

    Google Scholar 

  • Widrow B and Hoff ME Jr. (1960) Adaptive switching circuits. In: IRE WESCON Convention Record, pp. 96–104

  • PD. Wilde (1997) ArticleTitleThe magnitude of the diagonal elements in neural networks Neural Networks. 10 IssueID3 499–504 Occurrence Handle10.1016/S0893-6080(96)00094-9

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, 106, ROC

    Cheng-Yuan Liou

  2. M.D., Health Center, National Taiwan Normal University, Taipei, Taiwan, 106, ROC

    Shiao-Lin Lin

Authors
  1. Cheng-Yuan Liou
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shiao-Lin Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Cheng-Yuan Liou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liou, CY., Lin, SL. Finite Memory Loading in Hairy Neurons. Nat Comput 5, 15–42 (2006). https://doi.org/10.1007/s11047-004-5490-x

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-004-5490-x

Keywords