skip to main content
10.1145/3583780.3615143acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper

Assessing Student Performance with Multi-granularity Attention from Online Classroom Dialogue

Published: 21 October 2023 Publication History

Abstract

Accurately judging students' ongoing performance is very crucial for real-world educational scenarios. In this work, we focus on the task of automatically predicting students' levels of mastery of math questions from teacher-student classroom dialogue data in the online learning environment. We propose a novel neural network armed with a multi-granularity attention mechanism to capture the personalized pedagogical instructions from the very noisy teacher-student dialogue transcriptions. We conduct experiments on a real-world educational dataset and the results demonstrate the superiority and availability of our model in terms of various evaluation metrics.

References

[1]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings.
[2]
Nathaniel Blanchard, Michael Brady, Andrew M Olney, Marci Glaus, Xiaoyi Sun, Martin Nystrand, Borhan Samei, Sean Kelly, and Sidney D'Mello. 2015. A Study of Automatic Speech Recognition in Noisy Classroom Environments for Automated Dialog Analysis. In International Conference on Arti?cial Intelligence in Education. Springer, 23--33.
[3]
Jiahao Chen, Zitao Liu, and Weiqi Luo. 2022. Wide & Deep Learning for Judging Student Performance in Online One-on-One Math Classes. In International Conference on Artificial Intelligence in Education. Springer, 213--217.
[4]
Albert T Corbett and John R Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction, Vol. 4, 4 (1994), 253--278.
[5]
Yiming Cui, Ziqing Yang, and Xin Yao. 2023. Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca. arXiv preprint arXiv:2304.08177 (2023).
[6]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. 4171--4186.
[7]
Tejas I Dhamecha, Smit Marvaniya, Swarnadeep Saha, Renuka Sindhgatta, and Bikram Sengupta. 2018. Balancing human efforts and performance of student response analyzer in dialog-based tutors. In International Conference on Arti?cial Intelligence in Education. Springer, 70--85.
[8]
Yihuan Dong, Samiha Marwan, Preya Shabrina, Thomas Price, and Tiffany Barnes. 2021. Using Student Trace Logs to Determine Meaningful Progress and Struggle during Programming Problem Solving. International Educational Data Mining Society (2021).
[9]
Susan E Embretson and Steven P Reise. 2013. Item Response Theory. Psychology Press.
[10]
Ge Gao, Samiha Marwan, and Thomas W Price. 2021. Early Performance Prediction using Interpretable Patterns in Programming Process Data. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 342--348.
[11]
TAL Education Group. 2020. EduRoBERTa. https://github.com/tal-tech/edu-bert. Accessed: 2022-02-07.
[12]
Eeva Haataja, Enrique Garcia Moreno-Esteva, Visajaani Salonen, Anu Laine, Miika Toivanen, and Markku S Hannula. 2019. Teacher's visual attention when scaffolding collaborative mathematical problem solving. Teaching and Teacher Education, Vol. 86 (2019), 102877.
[13]
Qibin Hou, Daquan Zhou, and Jiashi Feng. 2021. Coordinate Attention for Efficient Mobile Network Design. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
[14]
Hang Li, Wenbiao Ding, Zhongqin Wu, and Zitao Liu. 2021. Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognition. In Interspeech 2021. ISCA.
[15]
Yang Li, Lukasz Kaiser, Samy Bengio, and Si Si. 2019. Area attention. In International Conference on Machine Learning. PMLR, 3846--3855.
[16]
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A Structured Self-attentive Sentence Embedding. In International Conference on Learning Representations.
[17]
Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, and Weiqi Luo. 2022a. simpleKT: A Simple But Tough-to-Beat Baseline for Knowledge Tracing. In The Eleventh International Conference on Learning Representations.
[18]
Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, Jiliang Tang, and Weiqi Luo. 2022b. pyKT: a python library to benchmark deep learning based knowledge tracing models. Advances in Neural Information Processing Systems, Vol. 35 (2022), 18542--18555.
[19]
Nora A McIntyre, Halszka Jarodzka, and Robert M Klassen. 2019. Capturing teacher priorities: Using real-world eye-tracking to investigate expert teacher priorities across two cultures. Learning and Instruction, Vol. 60 (2019), 215--224.
[20]
Mehrdad Mirzaei and Shaghayegh Sahebi. 2019. Modeling Students' Behavior Using Sequential Patterns to Predict Their Performance. In International Conference on Arti?cial Intelligence in Education. Springer, 350--353.
[21]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep Knowledge Tracing. Advances in Neural Information Processing Systems, Vol. 28 (2015).
[22]
Anna-Katharina Praetorius, Katharina Drexler, Lisa Rösch, Eva Christophel, Nora Heyne, Annette Scheunpflug, Horst Zeinz, and Markus Dresel. 2015. Judging students' self-concepts within 30 s? Investigating judgement accuracy in a zero-acquaintance situation. Learning and Individual Differences, Vol. 37 (2015), 231--236.
[23]
Shaghayegh Sahebi and Peter Brusilovsky. 2018. Student Performance Prediction by Discovering Inter-Activity Relations. International Educational Data Mining Society (2018).
[24]
Richard J Shavelson. 1978. Teachers' Estimates of Student 'States of Mind' and Behavior. Journal of Teacher Education, Vol. 29, 5 (1978), 37--40.
[25]
Kathleen Stürmer, Karen D Könings, and Tina Seidel. 2013. Declarative knowledge and professional vision in teacher education: Effect of courses in teaching and learning. British Journal of Educational Psychology, Vol. 83, 3 (2013), 467--483.
[26]
Cui Tang, Yuanxin Ouyang, Wenge Rong, Jingshuai Zhang, and Zhang Xiong. 2018. Time Series Model for Predicting Dropout in Massive Open Online Courses. In International Conference on Arti?cial Intelligence in Education. Springer, 353--357.
[27]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
[28]
Rurik Tywoniw, Scott A Crossley, Jaclyn Ocumpaugh, Shamya Karumbaiah, and Ryan Baker. 2020. Relationships Between Math Performance and Human Judgments of Motivational Constructs in an Online Math Tutoring System. In International Conference on Arti?cial Intelligence in Education. Springer, 329--333.
[29]
Vincent Van Asch. 2013. Macro-and micro-averaged evaluation measures. Belgium: CLiPS, Vol. 49 (2013).
[30]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical Attention Networks for Document Classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1480--1489.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
October 2023
5508 pages
ISBN:9798400701245
DOI:10.1145/3583780
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. ai in education
  2. assessment
  3. classroom dialogue
  4. student modeling

Qualifiers

  • Short-paper

Conference

CIKM '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 279
    Total Downloads
  • Downloads (Last 12 months)109
  • Downloads (Last 6 weeks)5
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media