Skip to main content
SpringerLink
Log in

Nonlinear quantization on Hebbian-type associative memories

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Hebbian-type associative memory is characterized by its simple architecture. However, the hardware implementation of Hebbian-type associative memories is normally complicated when there are a huge number of patterns stored. To simplify the interconnection values of a network, a nonlinear quantization strategy is presented. The strategy takes into account the property that the interconnection values are Gaussian distributed, and divides the interconnection weight values into a small number of unequal ranges accordingly. Interconnection weight values in each range contain information equally and each range is quantized to a value.

The equation of probability of direct convergence was derived. The probability of direct convergence of nonlinear quantized networks with a small number of ranges is compatible with their original networks. The effects of linear and nonlinear quantization were also assessed in terms of recall capability, information capacity, and number of bits storing interconnection values saved by quantization. The performance of the proposed nonlinear quantization strategy is better than that of the linear quantization while retaining a recall capability that is compatible with its original network. The proposed approach reduces the number of connection weights and the size of the chip areas of a Hebbian-type associative memory while approximately retaining its performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Verleysen M, Sirletti B (1989) A high-storage capacity content-addressable memory and its learning algorithm. IEEE Trans Circuits Syst 36(5):762–766

    Article  Google Scholar 

  2. Kamio T, Fujisaka H, Morisue M (2001) Backpropagation algorithm for LOGic oriented neural networks with quantized weights and multilevel threshold neurons. IEICE Trans Fundam E84-A:705–712

    Google Scholar 

  3. Heittmann A, Ruckert U (2002) Mixed mode VLSI implementation of a neural associative memory. Analog Integr Circuits Signal Process 30(2):159–172

    Article  Google Scholar 

  4. Kamio T, Fujisaka H, Morisue M (2002) Associative memories using interaction between multilayer perceptrons and sparsely interconnected neural networks. IEICE Trans Fundam E85-A:1220–1228

    Google Scholar 

  5. Kamio T, Morisue M (2003) A synthesis procedure for associative memories using cellular neural networks with space-invariant cloning template library. In: Proceedings of the international joint conference on neural networks, pp 885–890

    Chapter  Google Scholar 

  6. Ho C, Ling B, Lam HK, Nasir M (2008) Global convergence and limit cycle behavior of weights of perceptron. IEEE Trans Neural Netw 19(6):938–947

    Article  Google Scholar 

  7. Hopfield JJ, Tank DW (1985) Neural computation of decisions in optimization problems. Biol Cybern 52(3):141–152

    MathSciNet  MATH  Google Scholar 

  8. Tank DW, Hopfield JJ (1986) Simple optimization networks: A/D converter and a linear programming circuit. IEEE Trans Circuits Syst CAS-33:533–541

    Article  Google Scholar 

  9. Sussner P, ME Valle (2006) Gray-scale morphological associative memories. IEEE Trans Neural Netw 17(3):559–570

    Article  Google Scholar 

  10. Zhang H, Huang W, Huang Z, Zhang B (2005) A kernel autoassociator approach to pattern classification. IEEE Trans Syst Man Cybern, Part B, Cybern 35(3):593–606

    Article  Google Scholar 

  11. Sussner P, Valle ME (2007) Morphological and certain fuzzy morphological associative memories for classification and prediction. In: Kaburlasos V, Ritter G (eds) Computational intelligence based on lattice theory. Springer, Heidelberg, pp 149–172

    Chapter  Google Scholar 

  12. Costantini G, Casali D, Perfetti R (2003) Neural associative memory storing gray-coded gray-scale images. IEEE Trans Neural Netw 14(3):703–707

    Article  Google Scholar 

  13. Costantini G, Casali D, Perfetti R (2006) Associative memory design for 256 gray-level images using a multilayer neural network. IEEE Trans Neural Netw 17(2):519–522

    Article  Google Scholar 

  14. Lee DL (2006) Improvement of complex-valued Hopfield associative memory by using generalized projection rules. IEEE Trans Neural Netw 17(5):1341–1347

    Article  Google Scholar 

  15. Monteros Rave de los A, Azuela JHS (2008) A bidirectional hetero-associative memory for true-color patterns. Neural Process Lett 28(3):131–153

    Article  Google Scholar 

  16. Valle ME (2009) A class of sparsely connected autoassociative morphological memories for large color images. IEEE Trans Neural Netw 20(6):1045–1050

    Article  Google Scholar 

  17. Zheng P, Zhang J, Tang W (2010) Color image associative memory on a class of Cohen-Grossberg networks. Pattern Recognit 43(10):3255–3260

    Article  MATH  Google Scholar 

  18. Zhang BL, Zhang H, Ge SS (2004) Face recognition by applying wavelet subband representation and kernel associative memory. IEEE Trans Neural Netw 15(1):166–177

    Article  MathSciNet  Google Scholar 

  19. Zhang H, Zhang B, Huang W, Tian Q (2005) Gabor wavelet associative memory for face recognition. IEEE Trans Neural Netw 16(1):275–278

    Article  Google Scholar 

  20. Zhang D, Zuo W (2007) Computational intelligence-based biometric technologies. IEEE Comput Intell Mag 2(2):26–36

    Article  MathSciNet  Google Scholar 

  21. Han SJ, Oh SY (2009) A new paradigm for real-time parallel storage and recognition of patterns based on a hierarchical organization of associative memories utilizing Walsh function encoding. Appl Intell 31(3):305–317

    Article  Google Scholar 

  22. Amit DJ (1989) Modeling brain function: the world of attractor neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  23. Sompolinsky H (1987) The theory of neural networks: The Hebb rule and beyond. Heidelberg Colloquium on Glassy Dyn 275:485–527

    Article  MathSciNet  Google Scholar 

  24. Chung PC, Krile TF (1992) Characteristics of Hebbian-type associative memories having faulty interconnections. IEEE Trans Neural Netw 3(6):969–980

    Article  Google Scholar 

  25. Chung PC, Krile TF (1995) Reliability characteristics of quadratic Hebbian-type associative memories in optical and electronic network implementations. IEEE Trans Neural Netw 6(2):357–367

    Article  Google Scholar 

  26. Guzmán E, Alvarado S, Pogrebnyak O, Sánchez LP, Yánez C (2007) Hardware implementation of image recognition system based on morphological associative memories and discrete wavelet transform. LNCS, vol 4827. Springer, Berlin, pp 664–677

    Google Scholar 

  27. Guzmán E, Alvarado S, Pogrebnyak O, Yánez C (2007) Image recognition processor based on morphological associative memories. In: IEEE proceedings of the electronics, robotics and automotive mechanics conference, pp 260–265

    Google Scholar 

  28. Ahmadi A, Mattausch HJ, Abedin MA, Saeidi M, Koide T (2011) An associative memory-based learning model with an efficient hardware implementation in FPGA. Expert Syst Appl 38:3499–3513

    Article  Google Scholar 

  29. Szabó T, Horváth G (2004) An efficient hardware implementation of feed-forward neural networks. Appl Intell 21(2):143–158

    Article  MATH  Google Scholar 

  30. Jiang M, Gielen G (2008) Analysis of quantization effects on high-order function neural networks. Appl Intell 28(1):51–67

    Article  Google Scholar 

  31. Wang JH (1998) Principal interconnections in higher order Hebbian-type associative memories. IEEE Trans Knowl Data Eng 10(2):342–344

    Article  Google Scholar 

  32. Chung PC, Tsai CT, Sun YN (1994) Characteristics of Hebbian-type associative memories with quantized interconnections. IEEE Trans Circuits Syst 41(2):168–171

    MATH  Google Scholar 

  33. Davey N, Frank R, Hunt S, Adams R, Calcraft L (2004) High capacity associative memory models—binary and bipolar representation. In: Proceedings of the eighth IASTED international conference artificial intelligence and soft computing, Marbella, Spain, pp 392–397

    Google Scholar 

  34. Chung PC, Tsai CT, Sun YN (1994) Linear quantization of Hebbian-type associative memories in interconnection implementation. In: IEEE international conference on neural networks, Orlando, pp 1092–1097

    Google Scholar 

  35. McEliece RJ, Posner EC (1987) The capacity of the Hopfield associative memory. IEEE Trans Inf Theory 33(4):461–482

    Article  MathSciNet  MATH  Google Scholar 

  36. Willshaw DJ, Buneman OP, Longuet-Higgins HC (1969) Nonholographic associative memory. Nature 222:960–962

    Article  Google Scholar 

  37. Palm G, Sommer FT (1992) Information capacity in recurrent McCulloch–Pitts networks with sparsely coded memory states. Network 3(2):177–186

    Article  MATH  Google Scholar 

  38. Wang JH, Krile TF, Walkup JF (1990) Determination of Hopfield associative memory characteristics using a single parameter. Neural Netw 3(3):319–331

    Article  Google Scholar 

  39. Walkup JF, Krile TF (1991) Programmable optical quadratic neural networks. Technical Report, Department of Electrical Engineering, Texas Tech University

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Tunghai University, 181 Sec. 3, Taichung-Kong Rd., Taichung, 407, Taiwan

    Chishyan Liaw & Ching-Tsorng Tsai

  2. Department of Information Management, Hsiuping Institute of Technology, 11 Gungge Rd., Dali, Taichung, 412, Taiwan

    Chao-Hui Ko

Authors
  1. Chishyan Liaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Tsorng Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao-Hui Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chishyan Liaw.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liaw, C., Tsai, CT. & Ko, CH. Nonlinear quantization on Hebbian-type associative memories. Appl Intell 36, 824–833 (2012). https://doi.org/10.1007/s10489-011-0299-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-011-0299-7

Keywords

Search

Navigation