Abstract
Sparse Distributed Memory (SDM) and Binary Sparse Distributed Representations (BSDR) are phenomenological models of different aspects of biological memory. SDM as a neural network represents the functioning of noise and damage tolerant associative memory. BSDR represents methods of encoding holistic (structural) information in binary vectors. The idea of SDM- BSDR integration appeared long ago. However, SDM is inefficient in the role of BSDR cleaning memory. We can fill the gap between BSDR and SDM using the results of a 3rd theory related to sparse signals: Compressive Sensing (CS). An integrated semantic storage model is presented in this paper. It is called CS-SDM since it uses a new CS-based SDM design for cleaning memory applied to BSDR. CS-SDM implementation is based on GPU. The model’s capacity and denoising capabilities are significantly better than those of classical SDM designs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kanerva, P.: Sparse Distributed Memory. MIT Press, Cambridge, MA (1988)
Kanerva, P.: Sparse Distributed Memory and Related Models. Associative Neural Memories: Theory and Implementation, pp. 5–76. Oxford University Press, New York (1993)
Jaeckel, L.A.: An Alternative Design for a Sparse Distributed Memory. Report TR 89.28, Research Institute for Advanced Computer Science (RIACS), NASA Research Centre at Ames, pp. 13–20 (1993)
Jaeckel, L.A.: A Class of Designs for a Sparse Distributed Memory. Report TR 89.30, Research Institute for Advanced Computer Science (RIACS), NASA Research Centre at Ames, pp. 17–25 (1993)
Marr, D.A.: Theory of Cerebellar Cortex. The J. Physiol. 202(2), 437–470 (1969)
Smith, D.J., Forrest, S., Perelson, A.S.: Immunological Memory is Associative. Artificial Immune Systems and their Applications, pp. 105–112. Springer, Berlin (1993)
Sjödin, G.: The Sparchunk Code: A method to build higher-level structures in a sparsely encoded SDM. In: Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN/WCCI’98, pp. 50–58. Springer, London (1998)
Rachkovskij, D.A., Kussul, E.M.: Binding and normalization of binary sparse distributed representations by context-dependent thinning. Neural Comput. 13(2), 411–452 (2001)
Laiho M., Poikonen J.H., Kanerva P., Lehtonen E.: High dimensional computing with sparse vectors. In: IEEE Biomedical Circuits and Systems Conference “Engineering for Healthy Minds and Able Bodies” BioCAS-2015, pp. 1–4 (2015)
Schlegel, K, Neubert, P, Protzel, P.: A comparison of Vector Symbolic Architectures. arXiv:2001.11797v3 [cs.AI] https://arxiv.org/abs/2001.11797 (2020)
Gayler, R.W.: Vector symbolic architectures answer Jackendoff’s challenges for cognitive neuroscience. In: Proc. ICCS/ASCS International Conference on Cognitive Science, pp. 133–138. CogPrints, University of New South Wales, Sydney (2003)
Rachkovskij, D.A.: Codevectors: Sparse Binary Distributed Representations of Numerical Data, p. 200. Kyiv, Ukraine, Interservice (2019).(in Russian)
Candès, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59(8), 1207–1223 (2006)
Candès, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)
Dantzig, G.B.: Linear Programming and Extensions. Princeton University Press, Princeton, NJ (1963)
Needell, D., Tropp, J.A.: CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009)
Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. on Signal Processing 41(12), 3397–3415 (1993)
Open-source library for CoSaMP algorithm (2019). https://github.com/rfmiotto/CoSaMP/blob/master/cosamp.ipynb
Virtanen, P., et al.: SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17(3), 261–272 (2020)
Linear programming module from SciPy library (2020). https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html
Vdovychenko, R.O.: The computer program “Hybrid neural memory model CS-SDM”. Copyright #104882, May 26, 2021. Ukrainian Intellectual Property Institute (2021)
Open-source library CS-SDM (2022). https://github.com/Rolandw0w/phd-sdm-cs
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vdovychenko, R., Tulchinsky, V. (2023). Sparse Distributed Memory for Sparse Distributed Data. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 542. Springer, Cham. https://doi.org/10.1007/978-3-031-16072-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-16072-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16071-4
Online ISBN: 978-3-031-16072-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)