Skip to main content
Springer Nature Link
Log in

Automated Assessment of Comprehension Strategies from Self-explanations Using Transformers and Multi-task Learning

  • Conference paper
  • First Online:
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky (AIED 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1831))

Included in the following conference series:

Abstract

Self-explanation practice is an effective method to support students in better understanding complex texts. This study focuses on automatically assessing the comprehension strategies employed by readers while understanding STEM texts. Data from 3 datasets (N = 11,833) with self-explanations annotated on different comprehension strategies (i.e., bridging, elaboration, and paraphrasing) and an overall quality score was used to train various machine learning models in both single-task and multi-task setups. Our end-to-end neural architecture considers RoBERTa as an encoder applied to the target and self-explanation texts, combined with handcrafted features for assessing text cohesion and filtering out low-quality examples. The best configuration obtained a .699 weighted F1-score for the overall self-explanation quality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adedoyin, O.B., Soykan, E.: Covid-19 pandemic and online learning: the challenges and opportunities. In: Interactive Learning Environments, pp. 1–13 (2020)

    Google Scholar 

  2. Caruana, R.: Multitask learning. Mach. Learn. 28, 41–75 (1997)

    Article  Google Scholar 

  3. Dascalu, M., Crossley, S., McNamara, D., Dessus, P., Trausan-Matu, S.: Please ReaderBench This Text: A Multi-dimensional Textual Complexity Assessment Framework, pp. 251–271. Nova Science Publishers Inc., New York (2018)

    Google Scholar 

  4. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the NAACL, vol. 1, pp. 4171–4186 (2019)

    Google Scholar 

  5. Jurafsky, D., Martin, J.H.: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Pearson Prentice Hall, Upper Saddle River (2009)

    Google Scholar 

  6. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019), http://arxiv.org/abs/1907.11692

  7. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2018)

    Google Scholar 

  8. McNamara, D.S.: SERT: self-explanation reading training. Discourse Process. 38(1), 1–30 (2004)

    Article  MathSciNet  Google Scholar 

  9. McNamara, D.S., Levinstein, I.B., Boonthum, C.: iSTART: interactive strategy training for active reading and thinking. Behav. Res. Methods Instrum. Comput. 36(2), 222–233 (2004)

    Article  Google Scholar 

  10. McNamara, D.S., Magliano, J.: Toward a comprehensive model of comprehension. Psychol. Learn. Motiv. 51, 297–384 (2009)

    Article  Google Scholar 

  11. National Assessment of Educational Progress: The nation’s report card: mathematics and reading at U.S. department of education, institute of education sciences. National Center for Education Statistics (2022). https://www.nationsreportcard.gov/reading/nation/scores/

  12. Panaite, M., et al.: Bring It on! Challenges encountered while building a comprehensive tutoring system using ReaderBench. In: Penstein Rosé, C., et al. (eds.) AIED 2018. LNCS (LNAI), vol. 10947, pp. 409–419. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93843-1_30

    Chapter  Google Scholar 

  13. Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017). http://arxiv.org/abs/1706.03762

  14. Wolf, T., et al.: Transformers: State-of-the-art natural language processing. In: EMNLP (2020)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Ministry of Research, Innovation, and Digitalization, project CloudPrecis, Contract Number 344/390020/06.09.2021, MySMIS code: 124812, within POC, the Ministry of European Investments and Projects, POCU 2014–2020 project, Contract Number 62461/03.06.2022, MySMIS code: 153735, the IES (NSF R305A130124, R305A190063), the U.S. Department of Education, and the NSF (NSF REC0241144; IIS-0735682).

Author information

Authors and Affiliations

  1. University Politehnica of Bucharest, 313 Splaiul Independentei, 060042, Bucharest, Romania

    Bogdan Nicula, Marilena Panaite & Mihai Dascalu

  2. Fotonation SRL, Blvd. Timisoara, Nr. 4A, 061328, Bucharest, Romania

    Bogdan Nicula

  3. Department of Psychology, Arizona State University, PO Box 871104, Tempe, AZ, 85287, USA

    Tracy Arner & Danielle S. McNamara

  4. Math/CIS Department, SUNY at Old Westbury, Old Westbury, NY, 11568, USA

    Renu Balyan

  5. Academy of Romanian Scientists, Str. Ilfov, Nr. 3, 050044, Bucharest, Romania

    Mihai Dascalu

Authors
  1. Bogdan Nicula
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marilena Panaite
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tracy Arner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renu Balyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mihai Dascalu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Danielle S. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mihai Dascalu .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. University of Leeds, Leeds, UK

    Vania Dimitrova

  4. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  5. UNED, Madrid, Spain

    Olga C. Santos

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nicula, B., Panaite, M., Arner, T., Balyan, R., Dascalu, M., McNamara, D.S. (2023). Automated Assessment of Comprehension Strategies from Self-explanations Using Transformers and Multi-task Learning. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol 1831. Springer, Cham. https://doi.org/10.1007/978-3-031-36336-8_107

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36336-8_107

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36335-1

  • Online ISBN: 978-3-031-36336-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics