Skip to main content
SpringerLink
Log in

Combining fMRI Data and Neural Networks to Quantify Contextual Effects in the Brain

  • Conference paper
  • First Online:
Book cover Brain Informatics (BI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11309))

Included in the following conference series:

Abstract

Does word meaning change according to the context? Although this hypothesis has existed for a long time, only recently it has become possible to test it based on neuroimaging. Embodiment theories of knowledge representation suggest that word meaning consist of a collection of attributes defined in terms of various neural systems. This approach represents an unlimited number of objects through weighted attributes and the weights may change in context. This paper aims at quantifying such dynamic meanings using computational modeling. A neural network is trained with backpropagation to map attribute-based representations to fMRI images of subjects reading everyday sentences. Backpropagation is then extended to the features, demonstrating how they change in different sentence contexts for the same word. Indeed, statistically significant changes occurred across similar contexts and across different subjects, quantifying for the first time how attribute weightings for the same word are modified by context. Such dynamic representations of meaning could be used in future natural language processing systems, allowing them to mirror human performance more accurately.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Regier, T.: The Human Semantic Potential. MIT Press, Cambridge (1996)

    Google Scholar 

  2. Landau, B., Smith, L., Jones, S.: Object perception and object naming in early development. Trends Cogn. Sci. 27, 19–24 (1998)

    Article  Google Scholar 

  3. Barsalou, L.W.: Grounded cognition. Annl. Rev. Psyc. 59, 617–845 (2008)

    Article  Google Scholar 

  4. Binder, J.R., Desai, R.H., Graves, W.W., Conant, L.L.: Where is the semantic system? A critical review of 120 neuroimaging studies. Cereb. Cortex 19, 2767–2769 (2009)

    Article  Google Scholar 

  5. Binder, J.R., Desai, R.H.: The neurobiology of semantic memory. Trends Cogn. Sci. 15(11), 527–536 (2011)

    Article  Google Scholar 

  6. Binder, J.R., et al.: Toward a brain-based Comp. Sem. Cogn. Neuropsychol. 33(3–4), 130–174 (2016)

    Article  Google Scholar 

  7. Binder, J.R.: In defense of abstract conceptual representations. Psychon. Bull. Rev. 23, 1096–1108 (2016)

    Article  Google Scholar 

  8. Pecher, D., Zeelenberg, R., Barsalou, L.W.: Sensorimotors simulations underlie conceptual representations: modality-specific effects of prior activation. Psychon. Bull. Rev. 11, 164–167 (2004)

    Article  Google Scholar 

  9. Aguirre-Celis, N., Miikkulainen R.: From words to sentences & back: characterizing context-dependent meaning rep in the brain. In: Proceedings of the 39th Annual Meeting of the Cognitive Science Society, London, UK, pp. 1513–1518 (2017)

    Google Scholar 

  10. Glasgow, K., Roos, M., Haufler, A. J., Chevillet, M., A., Wolmetz, M.: Evaluating semantic models with word-sentence relatedness. arXiv:1603.07253 (2016)

  11. Anderson, A.J., et al.: Perdicting Neural activity patterns associated with sentences using neurobiologically motivated model of semantic representation. Cereb. Cortex 1–17 (2016). https://doi.org/10.1093/cercor/bhw240

  12. Burgess, C.: From simple associations to the building blocks of language: modeling meaning with HAL. Behav. Res. Methods Inst. Com. 30, 188–198 (1998)

    Article  Google Scholar 

  13. Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: the latent semantic analysis theory. Psychol. Rev. 104, 211–240 (1997)

    Article  Google Scholar 

  14. Vinyals, O., Toshev, A., Bengio, S., Erham, D.: Show and tell: a new image caption generator. arXiv:1506.03134v2 (2015)

  15. Miikkulainen, R., Dyer, M.G.: Natural language processing with modular PDP networks and distributed lexicon. Cogn. Sci. 15, 343–399 (1991)

    Article  Google Scholar 

  16. Estes, Z., Golonka, S., Jones, L.L.: Thematic thinking: the apprehension and consequences of thematic relations. Psychol. Learn. Motiv. 54, 249–294 (2011)

    Article  Google Scholar 

  17. Anderson, A.J., et al.: Multiple regions of a cortical network commonly encode the meaning of words in multiple grammatical positions of read sentences. Cereb. Cortex 1–16 (2018). https://doi.org/10.1093/cercor/bhy110

  18. Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cogn. Sci. 38(8), 1388–1439 (2010). https://doi.org/10.1111/j.1551-6709.2010.01106.x

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Jeffery Binder (Medical College of Wisconsin), Rajeev Raizada and Andrew Anderson (University of Rochester), Mario Aguilar and Patrick Connolly (Teledyne Scientific Company) for providing this data and insight for this research. This work was supported in part by IARPA-FA8650-14-C-7357 and by NIH 1U01DC014922 grants.

Author information

Authors and Affiliations

  1. ITESM, E. Garza Sada 2501, Monterrey, NL, 64840, Mexico

    Nora Aguirre-Celis

  2. The University of Texas at Austin, 2317 Speedway, Austin, TX, 78712, USA

    Nora Aguirre-Celis & Risto Miikkulainen

Authors
  1. Nora Aguirre-Celis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Risto Miikkulainen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nora Aguirre-Celis .

Editor information

Editors and Affiliations

  1. University of Texas at Arlington, Arlington, TX, USA

    Shouyi Wang

  2. University of Southern California, West Hollywood, USA

    Vicky Yamamoto

  3. Department of Mathematics, University of Texas at Arlington, Arlington, TX, USA

    Jianzhong Su

  4. Maebashi Institute of Technology, Gunma, Japan

    Yang Yang

  5. The University of Texas at Arlington, Arlington, USA

    Erick Jones

  6. Louisiana Tech University, Arlington, TX, USA

    Leon Iasemidis

  7. Carnegie Mellon University, Pittsburgh, PA, USA

    Tom Mitchell

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Aguirre-Celis, N., Miikkulainen, R. (2018). Combining fMRI Data and Neural Networks to Quantify Contextual Effects in the Brain. In: Wang, S., et al. Brain Informatics. BI 2018. Lecture Notes in Computer Science(), vol 11309. Springer, Cham. https://doi.org/10.1007/978-3-030-05587-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05587-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05586-8

  • Online ISBN: 978-3-030-05587-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation