Skip to main content

Advertisement

Springer Nature Link
Log in

How internal neurons represent the short context: an emergent perspective

  • Regular Paper
  • Published:
Progress in Artificial Intelligence Aims and scope Submit manuscript

Abstract

Natural language acquisition is a crucial research domain of artificial intelligence. To enable the computer acquire and understand the human language and generate the corresponding response, we must research the principle of language acquiring of human being. Most of the prior language acquisition methods use handcrafted internal representation which is not sufficiently brain-based. An emergent developmental network (DN) is presented to acquire the certain speech extracted by mel-frequency cepstrum coefficient (MFCC), from sensory and motor experience. This work is different in the sense that we focused on mechanisms that enable a system to develop its emergent representations from its operational experience. In this work, internal unsupervised neurons of the DN are used to represent the short contexts, and the competitions among the internal neurons enable them to represent different short contexts. To demonstrate the acquisition effect, we study and analyze the influences of different network structure (i.e., different neuron number and weight threshold) on the language acquisition rate. Four speech acquisition experiments demonstrate efficiently how such internal neurons represent the short context while they are not directly supervised by the external environment. The presented network is developmental which means that the internal representations are directly learned from the signals of the input and motor ports, not designed internally for particular task, hence the same learning principles are potentially suitable for other sensory modalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Redington, M., Chater, N.: Connectionist and statistical approaches to language acquisition: a distributional perspective. Lang. Cognit. Process. 13(2/3), 129–191 (1998)

    Article  Google Scholar 

  2. Gervain, J.: Plasticity in early language acquisition: the effects of prenatal and early childhood experience. Curr. Opin. Neurobiol. 35, 13–20 (2015)

    Article  Google Scholar 

  3. Chomsky, N.: The generative enterprise revisited. Mouton de Gruyter, Berlin (2004)

    Book  Google Scholar 

  4. Chomsky, N.: The minimalist program. MIT Press, Cambridge, MA (1995)

    MATH  Google Scholar 

  5. Chomsky, N.: Lectures on government and binding. Foris, Dordrecht (1981)

    Google Scholar 

  6. Pinker, S.: (2011) The Language Instinct: How the Mind Creates Language. William Morrow, New York, 1994

  7. Edelman, S., Waterfall, H.: Behavioral and computational aspects of language and its acquisition. Phys. Life Rev. 4, 253–277 (2007)

    Article  Google Scholar 

  8. Hawkins, R.: The nativist perspective on second language acquisition. Lingua 118, 465–477 (2008)

    Article  Google Scholar 

  9. Adger, D.: Core Syntax: A Minimalist Approach. Oxford University Press, New York (2003)

    Google Scholar 

  10. Anjomshoa, L., Sadighi, F.: Minimalism, economy, simplicity, and children language acquisition. Proc. Soc. Behav. Sci. 192, 142–149 (2015)

    Article  Google Scholar 

  11. Peregoy, D.: Behaviorist B.F. Skinner and Theory Research. Technical Report, Towson University, Baltimore, Maryland, November 20 (2012)

  12. Desai, R.: A model of frame and verb compliance in language acquisition. Neurocomputing 70, 2273–2287 (2007)

    Article  Google Scholar 

  13. Howell, S.R., Jankowicz, D., Becker, S.: A model of grounded language acquisition: sensorimotor features improve lexical and grammatical learning. J. Memory Lang. 53, 258–276 (2005)

    Article  Google Scholar 

  14. Hayes, S.C., Barnes-Holmes, D., Roche, B.: Relational Frame Theory: A post-Skinnerian Account of Human Language and Cognition. Plenum Press, New York, USA (2001)

    Google Scholar 

  15. Takac, M., Benuskova, L., Knott, A.: Mapping sensorimotor sequences to word sequences: a connectionist model of language acquisition and sentence generation. Cognition 125, 288–308 (2012)

    Article  Google Scholar 

  16. Piaget, J.: The language and thought of the child. The Essential Piaget. Basic Books Inc. (1977)

  17. Tomasello, M.: Do young children have adult syntactic competence? Cognition 74, 209–253 (2000)

    Article  Google Scholar 

  18. MacWhinney, B.: The Emergence of Language. Lawrence Erlbaum Associates, Mahwah, New Jersey, USA (1999)

    Google Scholar 

  19. MacWhinney, B.: How mental models encode embodied linguistic perspectives. Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA (2008)

    Google Scholar 

  20. Lacerda, F., Klintfors, E., Gustavsson, L., Lagerkvist, L., Marklund, E., Sundberg, U.: Ecological Theory of Language Acquisition. In: Proceedings of the Fourth International Workshop on Epigenetic Robotics, Genoa, Italy, August 25–27 (2004)

  21. Evans, V., Green, M.: Cognitive Linguistics: An Introduction, Hillsdale. Lawrence Erlbaum Associates, New Jersey (2006)

    Google Scholar 

  22. Tahriri, A.: Revisiting first language acquisition through empirical and rational perspectives. Online Subm. 11, 1–11 (2012)

    Google Scholar 

  23. Tomasello, M.: Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press, Cambridge, MA (2003)

    Google Scholar 

  24. Glenberg, A.M., Gallese, V.: Action-based language: a theory of language acquisition, comprehension, and production. Cortex 48, 905–922 (2012)

    Article  Google Scholar 

  25. Behjat, F.: Abandoning here-and-now language: sensorimotor development in the acquisition of Persian as a first language. Proc. Soc. Behav. Sci. 84, 1417–1420 (2013)

    Article  Google Scholar 

  26. Hsu, A.S., Chater, N., Vitanyi, P.M.B.: The probabilistic analysis of language acquisition: theoretical, computational, and experimental analysis. Cognition 120, 380–390 (2011)

    Article  Google Scholar 

  27. Hsu, A., Chater, N.: The logical problem of language acquisition goes probabilistic: no negative evidence as a window on language acquisition. Cognit. Sci. 34, 972–1016 (2010)

    Article  Google Scholar 

  28. Schilhab, T.: Doubletalk C The biological and social acquisition of language. Biol. Inspired Cognit. Arch. 13, 1–8 (2015)

    Google Scholar 

  29. Wintner, S.: Computational models of language acquisition. Comput. Ling. Intell. Text Process. 6008, 86–99 (2010)

    Article  Google Scholar 

  30. Lieven, E., Salomo, D., Tomasello, M.: Two-year-old childrens production of multiword utterances: a usage-based analysis. Cognit. Ling. 20(3), 481–507 (2009)

    Google Scholar 

  31. Freudenthal, D., Pine, J.M., Gobet, F.: Simulating the referential properties of Dutch, German, and English root infinitives in MOSAIC. Lang. Learning Develop. 5, 1–29 (2009)

    Article  Google Scholar 

  32. Sagae, K., Davis, E., Lavie, A., MacWhinney, B., Wintner, S.: Morphosyntactic annotation of CHILDES transcripts. J. Child Lang. 37(3), 705–729 (2010)

    Article  Google Scholar 

  33. Rasanen, O.: Computational modeling of phonetic and lexical learning in early language acquisition: existing models and future directions. Speech Commun. 54, 975–997 (2012)

    Article  Google Scholar 

  34. Jansen, A., Dupoux, E., Goldwater, S.: A Summary of the 2012 JHU CLSPWORKSHOP on zero resource speech technologies and models of early language acquisition. In: IEEE International Conference on Acoustics Speech and Signal Processing, Vancouver, British Columbia, Canada, May 26–31 (2013)

  35. Buttery, P.J.: Computational models for first language acquisition. Technical Report, UCAM-CL-TR-675. University of Cambridge, Computer Laboratory (2006)

  36. Weng, J.: Natural and Artificial Intelligence: Introduction to computational brain-mind. BMI press, Okemos, Michigan, USA (2012)

    Google Scholar 

  37. Guo, Q., Wu, X., Weng, J.: WWN-9: Cross-Domain Synaptic Maintenance and Its Application to Object Groups Recognition. In: 2014 International Joint Conference on Neural Networks (IJCNN), Beijing, China, July 6–11 (2014)

  38. Weng, J.: Why have we passed neural networks no not abstract well. Nat. Intell. INNS Mag. 1, 13–22 (2011)

    Google Scholar 

  39. Norouzian, A., Rose, R.: Facilitating open vocabulary spoken term detection using a multiple pass hybrid search algorithm. In: Proceedings of 2012 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Kyoto, Japan, March 25–30 (2012)

  40. Friederici, A.D.: The brain basis of language processing: from structure to function. Physiol. Rev. 91, 1357–1392 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors want to thank Professor Juyang Weng with department of computer science and engineering, Michigan State University, MI, USA, for much of the work was done when the first author was a visiting scholar in Michigan State University. This work is supported by the National Nature Science Funds of China with Grant No. 61174085.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University, Zhengzhou, 450001, Henan, China

    Dongshu Wang & Jiaming Chen

  2. Department of Research, The People’s Bank of China, Zhengzhou Sub-Branch, Zhengzhou, 450040, Henan, China

    Lei Liu

Authors
  1. Dongshu Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jiaming Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Dongshu Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Chen, J. & Liu, L. How internal neurons represent the short context: an emergent perspective. Prog Artif Intell 6, 67–77 (2017). https://doi.org/10.1007/s13748-016-0106-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13748-016-0106-0

Keywords