Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computational Collective Intelligence

ICCCI 2020: Computational Collective Intelligence pp 779–791Cite as

Towards Learning to Read Like Humans

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12496))

Abstract

Previous NLP works were successful in using human gaze behavior to improve task performance on their data sets. However, having to repeatedly collect gaze data for every new data set is impractical. Thus, there is a need for a method that will allow the utilization of available gaze data without the overhead. Our work presents a novel attempt to directly predict gaze features for each word with respect to its sentence. We take on a multi-corpus and task-agnostic approach: using four different eye-tracking data sets, regardless of reading task, material, and experiment design. Using only the word sequence as input to a 2-layer bidirectional LSTM, we achieve \(R^2\) scores in the range of 76.80 to 95.59 for the following five gaze features: Number of Fixations (NFIX), First Fixation Duration (FFD), Total Reading Time (TRT), Go-Past Time (GPT), and Gaze Duration (GD). In addition, we use the model to predict gaze features for words in seen and unseen sentences in an attempt to improve performance in two NLP tasks. This led to a slight increase in performance, supporting the potential of such a model. Our paper presents an exploratory experiment into this methodology.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R.: FLAIR: an easy-to-use framework for state-of-the-art NLP. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pp. 54–59. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019

    Google Scholar 

  2. Akbik, A., Blythe, D., Vollgraf, R.: Contextual string embeddings for sequence labeling. In: COLING 2018, 27th International Conference on Computational Linguistics, pp. 1638–1649 (2018)

    Google Scholar 

  3. Barrett, M., Bingel, J., Hollenstein, N., Rei, M., Søgaard, A.: Sequence classification with human attention. In: Proceedings of the 22nd Conference on Computational Natural Language Learning, pp. 302–312. Association for Computational Linguistics, Brussels, Belgium, October 2018

    Google Scholar 

  4. Cop, U., Dirix, N., Drieghe, D., Duyck, W.: Presenting geco: an eyetracking corpus of monolingual and bilingual sentence reading. Behav. Res. Methods 49(2), 602–615 (2017)

    Article  Google Scholar 

  5. Frank, S.L., Fernandez Monsalve, I., Thompson, R.L., Vigliocco, G.: Reading time data for evaluating broad-coverage models of english sentence processing. Behav. Res. Methods 45(4), 1182–1190 (2013)

    Article  Google Scholar 

  6. González-Garduño, A., Søgaard, A.: Learning to predict readability using eye-movement data from natives and learners (2018)

    Google Scholar 

  7. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  8. Hollenstein, N., Barrett, M., Troendle, M., Bigiolli, F., Langer, N., Zhang, C.: Advancing NLP with cognitive language processing signals http://arxiv.org/abs/1904.02682

  9. Hollenstein, N., Rotsztejn, J., Troendle, M., Pedroni, A., Zhang, C., Langer, N.: ZUCO, a simultaneous EEG and eye-tracking resource for natural sentence reading. Scientific Data 5, 180291 EP - (12 2018)

    Google Scholar 

  10. Hollenstein, N., Zhang, C.: Entity recognition at first sight: Improving NER with eye movement information. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 1–10. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019

    Google Scholar 

  11. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  12. Luke, S.G., Christianson, K.: The provo corpus: a large eye-tracking corpus with predictability norms. Behav. Res. Methods 50(2), 826–833 (2018)

    Article  Google Scholar 

  13. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS 2013, vol. 2, pp. 3111–3119. Curran Associates Inc., USA (2013)

    Google Scholar 

  14. Mishra, A., Dey, K., Bhattacharyya, P.: Learning cognitive features from gaze data for sentiment and sarcasm classification using convolutional neural network. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 377–387. Association for Computational Linguistics, Vancouver, Canada, July 2017

    Google Scholar 

  15. Mishra, A., Kanojia, D., Nagar, S., Dey, K., Bhattacharyya, P.: Harnessing cognitive features for sarcasm detection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1095–1104. Association for Computational Linguistics, Berlin, Germany, August 2016

    Google Scholar 

  16. Mishra, A., Kanojia, D., Nagar, S., Dey, K., Bhattacharyya, P.: Leveraging cognitive features for sentiment analysis. In: Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 156–166. Association for Computational Linguistics, Berlin, Germany, August 2016

    Google Scholar 

  17. Mishra, A., Tamilselvam, S., Dasgupta, R., Nagar, S., Dey, K.: Cognition-cognizant sentiment analysis with multitask subjectivity summarization based on annotators’ gaze behavior (2018)

    Google Scholar 

  18. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar, October 2014

    Google Scholar 

  19. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372 (1998)

    Article  Google Scholar 

  20. Singh, A.D., Mehta, P., Husain, S., Rajakrishnan, R.: Quantifying sentence complexity based on eye-tracking measures. In: Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pp. 202–212. The COLING 2016 Organizing Committee, Osaka, Japan, December 2016

    Google Scholar 

  21. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  22. Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CONLL-2003 shared task: Language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003 - Volume 4, CONLL 2003, pp. 142–147. Association for Computational Linguistics, Stroudsburg (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of the Philippines, Quezon City, Philippines

    Louise Gillian Bautista & Prospero Naval Jr.

Authors
  1. Louise Gillian Bautista
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prospero Naval Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Louise Gillian Bautista .

Editor information

Editors and Affiliations

  1. Department of Applied Informatics, Wrocław University of Science and Technology, Wroclaw, Poland

    Ngoc Thanh Nguyen

  2. Thua Thien Hue Center of Information Technology, Hue, Vietnam

    Bao Hung Hoang

  3. Vietnam - Korea University of Information and Communication Technology, University of Da Nang, Da Nang, Vietnam

    Cong Phap Huynh

  4. Department of Computer Engineering, Yeungnam University, Gyeungsan, Korea (Republic of)

    Dosam Hwang

  5. Department of Applied Informatics, Wrocław University of Science and Technology, Wroclaw, Poland

    Bogdan Trawiński

  6. Department of Information Systems, University of Münster, Münster, Germany

    Gottfried Vossen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bautista, L.G., Naval, P. (2020). Towards Learning to Read Like Humans. In: Nguyen, N.T., Hoang, B.H., Huynh, C.P., Hwang, D., Trawiński, B., Vossen, G. (eds) Computational Collective Intelligence. ICCCI 2020. Lecture Notes in Computer Science(), vol 12496. Springer, Cham. https://doi.org/10.1007/978-3-030-63007-2_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63007-2_61

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63006-5

  • Online ISBN: 978-3-030-63007-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation