Abstract
Recent models of associative long term memory (LTM) have emerged in the field of neuro-inspired computing. These models have interesting properties of error correction, robustness, storage capacity and retrieval performance. In this context, we propose a connectionist model of written word recognition with correction properties, using associative memories based on neural cliques. Similarly to what occurs in human language, the model takes advantage of the combination of phonological and orthographic information to increase the retrieval performance in error cases. Therefore, the proposed architecture and principles of this work could be applied to other neuro-inspired problems that involve multimodal processing, in particular for language applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
This work was supported by the European Research Council under ERC Grant agreement number 290901 NEUCOD.
- 3.
LIA_PHON v1.2, under GPL license, available in http://lia.univ-avignon.fr/chercheurs/bechet/download_fred.html.
- 4.
Example: if \(b=3\) the same letter pattern is activated in 3 adjacent clusters. Using the example of Fig. 3 activated fanals are {(B,C5);(B,C1);(B,C2);(R,C1);(R,C2);(R,C3);(A,C2);(A,C3);(A,C4);(I,C3);(I,C4);(I,C5);(N,C4);(N,C5);(N,C6)}.
- 5.
\(b_{0}=1\) and then \(b_{t+1}=2*b_{t}+1\) (for \(t=0,1,2,3,...\)) until the stopping condition is reached or \(b>c\).
- 6.
Lexique.org is a French lexical database of lexical information of 135,000 words and 55,000 lemmas.
- 7.
Accentuated and special characters are included.
- 8.
\(matching=numberCorrectFanals/c\).
- 9.
It provides zero if \(numberActivatedFanals=c\) and \(matching=1\) else it provides one.
- 10.
The English benchmark has 107 words, among which there are 73 unique words.
- 11.
https://gitlab.com/msobroza/context-typo-network.git.
- 12.
The reinforcement of connections in a multilayer clique-based neural network is an unpublished problem.
References
Aboudib, A., Gripon, V., Jiang, X.: A study of retrieval algorithms of sparse messages in networks of neural cliques. In: COGNITIVE 2014: The 6th International Conference on Advanced Cognitive Technologies and Applications (2014)
Azevedo, F.A.C., Carvalho, L.R.B., Grinberg, L.T., Farfel, J.M., Ferretti, R.E.L., Leite, R.E.P., Lent, R., Herculano-Houzel, S.: Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain. J. Comp. Neurol. 5, 532–541 (2009)
Berrou, C., Dufor, O., Gripon, V., Xiaoran, J.: Information, noise, coding, modulation: what about the brain? In: 8th International Symposium on Turbo Codes and Iterative Information Processing (ISTC) (2014)
Boguslawski, B., Gripon, V., Seguin, F., Heitzmann, F.: Huffman coding for storing non-uniformly distributed messages in networks of neural cliques. In: 28th Conference on Artificial Intelligence, vol. 1. AAAI (2014)
Christianson, K., Johnson, R., Rayner, K.: Letter transpositions within and across morphemes. J. Exp. Psychol. Learn. Mem. Cogn. 31(6), 1327 (2005)
Coltheart, M., Rastle, K., Perry, C., Langdon, R., Ziegler, J.: DRC: a dual route cascaded model of visual word recognition and reading aloud. Psychol. Rev. 108(1), 204 (2001)
Grainger, J., Granier, J., Farioli, F., Van Assche, E., Van Heuven, W.: Letter position information and printed word perception: the relative-position priming constraint. J. Exp. Psychol. Hum. Percep. Perform. 32(4), 865 (2006)
Grainger, J., Whitney, C.: Does the huamn mnid raed wrods as a wlohe? Trends Cogn. Sci. 8(2), 58–59 (2004)
Gripon, V., Berrou, C.: Sparse neural networks with large learning diversity. IEEE Trans. Neural Netw. 22(7), 1087–1096 (2011)
Gripon, V., Jiang, X.: Mémoires associatives pour observations floues. In: Proceedings of XXIV-th Gretsi Seminar (2013)
Jiang, X., Gripon, V., Berrou, C., Rabbat, M.: Storing sequences in binary tournament-based neural networks. IEEE Trans. Neural Netw. Learn. Syst. 27(5), 913–925 (2015a)
Jiang, X., Sobroza Marques, M.R., Kirsch, P.-J., Berrou, C.: Improved retrieval for challenging scenarios in clique-based neural networks. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2015. LNCS, vol. 9094, pp. 400–414. Springer, Cham (2015b). https://doi.org/10.1007/978-3-319-19258-1_34
Leaver, A., Van Lare, J., Zielinski, B., Halpern, A., Rauschecker, J.: Brain activation during anticipation of sound sequences. J. Neurosci. 29(8), 2477–2485 (2009)
McCulloch, W., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)
Mountcastle, V.: The columnar organization of the neocortex. Brain 120(4), 701–722 (1997)
Perea, M., Carreiras, M.: Do transposed-letter similarity effects occur at a prelexical phonological level? Q. J. Exp. Psychol. 59(9), 1600–1613 (2006)
Perea, M., Lupker, S., Kinoshita, S.: Transposed-letter confusability effects in masked form priming. Masked Priming: State of the Art, 97–120 (2003)
Plaut, D.: Connectionist approaches to reading. The Science of Reading: A Handbook, pp. 24–38 (2005)
Rayner, K., White, S., Johnson, R., Liversedge, S.: Raeding wrods with jubmled lettres there is a cost. Psychol. Sci. 17(3), 192–193 (2006)
Schacter, D.L., Buckner, R.L.: Priming and the brain. J. Neuron 20(2), 185–195 (1998)
Schoonbaert, S., Grainger, J.: Letter position coding in printed word perception: effects of repeated and transposed letters. Lang. Cogn. Process. 19(3), 333–367 (2004)
Starzyk, J., He, H., et al.: Spatio-temporal memories for machine learning: a long-term memory organization. IEEE Trans. Neural Netw. 20(5), 768–780 (2009)
Squire, L.R.: Memory systems of the brain: a brief history and current perspective. Neurobiol. Learn. Mem. 82(3), 171–177 (2004)
Tulving, E., Schacter, D.L.: Priming and human memory systems. JSTOR (1990)
Thivierge, J., Marcus, G.: The topographic brain: from neural connectivity to cognition. Trends Neurosci. 30(6), 251–259 (2007)
Van den Heuvel, M., Sporns, O.: Network hubs in the human brain. Trends Cogn. Sci. 17(12), 683–696 (2013)
Van Orden, G.: A rows is a rose: spelling, sound, and reading. Mem. Cogn. 15(3), 181–198 (1987)
Wang, W., Subagdja, B., Tan, A., Starzyk, J., et al.: Neural modeling of episodic memory: encoding, retrieval, and forgetting. IEEE Trans. Neural Netw. Learn. Syst. 23(10), 1574–1586 (2012)
Welvaert, M., Farioli, F., Grainger, J.: Graded effects of number of inserted letters in superset priming. Exp. Psychol. 55(1), 54–63 (2008)
Whitney, C.: How the brain encodes the order of letters in a printed word: the seriol model and selective literature review. Psychon. Bull. Rev. 8(2), 221–243 (2001)
Willshaw, D., Buneman, O., Longuet-Higgins, H.: Non-holographic associative memory. Nature (1969)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Marques, M.R.S., Jiang, X., Dufor, O., Berrou, C., Kim-Dufor, DH. (2018). A Connectionist Model of Reading with Error Correction Properties. In: Vetulani, Z., Mariani, J., Kubis, M. (eds) Human Language Technology. Challenges for Computer Science and Linguistics. LTC 2015. Lecture Notes in Computer Science(), vol 10930. Springer, Cham. https://doi.org/10.1007/978-3-319-93782-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-93782-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93781-6
Online ISBN: 978-3-319-93782-3
eBook Packages: Computer ScienceComputer Science (R0)