Abstract
We present a recurrent learning system that can incrementally integrate stimuli in two modalities, visual and auditory. The system consists of five self-organizing modules, each mapping input stimuli into respective latent spaces. Two sensory modules convert the input stimuli into an internal 3-D “neuronal code”. The central module integrates the bimodal information, and through modulatory top-down feedback influences the organization of data in two unimodal association units. Two feedback gains control the strength of the feedback connection. As an example we selected a set of Chinese characters and related spoken words. It is shown that the learning system can build a stable neuronal structure for incrementally applied visual and auditory stimuli.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Atteveldt, N.M.: Speech Meets Script: FMRI Studies on the Integration of Letters and Speech Sounds. University of Maastricht (2006)
Bookheimer, S.: How the brain reads Chinese characters. Neuroreport 12(1) (January 2001)
Calvert, G.A., Brammer, M.J., Iversen, S.D.: Crossmodal identification. Current Biology 2, 247–253 (1998)
Chou, S., Papliński, A.P., Gustafsson, L.: Speaker-dependent bimodal integration of Chinese phonemes and letters using multimodal self-organizing networks. In: Proc. Int. Joint Conf. Neural Networks, Orlando, Florida, pp. 248–253 (August 2007)
Dehaene, S.: Reading in the Brain. Viking (2009), http://pagesperso-orange.fr/readinginthebrain/figures.htm
Druckmann, S., Chklovskii, D.B.: Over-complete representations on recurrent neural networks can support persistent percepts. Advances in Neural Information Processing Systems (2010)
Fitch, W.T.: The Evolution of Language. Cambridge University Press (2010)
Jantvik, T., Gustafsson, L., Papliński, A.P.: A self-organized artificial neural network architecture for sensory integration with applications to letter–phoneme integration. Neural Computation 23, 2101–2139 (2011)
Kohonen, T.: Self-Organising Maps, 3rd edn. Springer, Berlin (2001)
Papliński, A.P.: Rotation invariant categorization of visual objects using Radon transform and self-organizing modules. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) ICONIP 2010, Part II. LNCS, vol. 6444, pp. 360–366. Springer, Heidelberg (2010)
Papliński, A.P.: Incremental self-organizing map (iSOM) in categorization of visual objects. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds.) ICONIP 2012, Part II. LNCS, vol. 7664, pp. 125–132. Springer, Heidelberg (2012)
Papliński, A.P.: Rotation invariant categorization of colour images using Radon transform. In: Proc. WCCI–IJCNN, pp. 1408–1413. IEEE (2012)
Papliński, A.P.: The angular integral of the radon transform (aniRT) as a feature vector in categorization of visual objects. In: Guo, C., Hou, Z.-G., Zeng, Z. (eds.) ISNN 2013, Part I. LNCS, vol. 7951, pp. 523–531. Springer, Heidelberg (2013)
Papliński, A.P., Gustafsson, L., Mount, W.M.: A model of binding concepts to spoken names. Aust. Journal of Intelligent Information Processing Systems 11(2), 1–5 (2010)
Papliński, A.P., Gustafsson, L., Mount, W.M.: A recurrent multimodal network for binding written words and sensory-based semantics into concepts. In: Lu, B.-L., Zhang, L., Kwok, J. (eds.) ICONIP 2011, Part I. LNCS, vol. 7062, pp. 413–422. Springer, Heidelberg (2011)
Siok, W., Fletcher, P.: The role of phonological awareness and visual orthographic skills in Chinese reading acquisition. Developmental Psychology 37(6), 886–889 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Papliński, A.P., Mount, W.M. (2013). Bimodal Incremental Self-Organizing Network (BiSON) with Application to Learning Chinese Characters. In: Lee, M., Hirose, A., Hou, ZG., Kil, R.M. (eds) Neural Information Processing. ICONIP 2013. Lecture Notes in Computer Science, vol 8226. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42054-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-42054-2_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42053-5
Online ISBN: 978-3-642-42054-2
eBook Packages: Computer ScienceComputer Science (R0)