skip to main content
10.1145/3242969.3264984acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Predicting Engagement Intensity in the Wild Using Temporal Convolutional Network

Authors Info & Claims
Published:02 October 2018Publication History

ABSTRACT

Engagement is the holy grail of learning whether it is in a classroom setting or an online learning platform. Studies have shown that engagement of the student while learning can benefit students as well as the teacher if the engagement level of the student is known. It is difficult to keep track of the engagement of each student in a face-to-face learning happening in a large classroom. It is even more difficult in an online learning platform where, the user is accessing the material at different instances. Automatic analysis of the engagement of students can help to better understand the state of the student in a classroom setting as well as online learning platforms and is more scalable. In this paper we propose a framework that uses Temporal Convolutional Network (TCN) to understand the intensity of engagement of students attending video material from Massive Open Online Courses (MOOCs). The input to the TCN network is the statistical features computed on 10 second segments of the video from the gaze, head pose and action unit intensities available in OpenFace library. The ability of the TCN architecture to capture long term dependencies gives it the ability to outperform other sequential models like LSTMs. On the given test set in the EmotiW 2018 sub challenge-"Engagement in the Wild", the proposed approach with Dilated-TCN achieved an average mean square error of 0.079.

References

  1. Rick D Axelson and Arend Flick. 2010. Defining student engagement. Change: The magazine of higher learning Vol. 43, 1 (2010), 38--43.Google ScholarGoogle Scholar
  2. Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson. 2015. Cross-dataset learning and person-specific normalisation for automatic action unit detection. In Automatic Face and Gesture Recognition (FG), 2015 11th IEEE International Conference and Workshops on, Vol. Vol. 6. IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  3. Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2013. Constrained local neural fields for robust facial landmark detection in the wild Proceedings of the IEEE International Conference on Computer Vision Workshops. 354--361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Tadas Baltrušaitis, Peter Robinson, and Louis-Philippe Morency. 2016. Openface: an open source facial behavior analysis toolkit Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 1--10.Google ScholarGoogle Scholar
  5. Jonathan Bidwell and Henry Fuchs. 2011. Classroom analytics: Measuring student engagement with automated gaze tracking. Behav Res Methods Vol. 49 (2011), 113.Google ScholarGoogle Scholar
  6. Mathieu Chollet and Stefan Scherer. 2017. Assessing public speaking ability from thin slices of behavior 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, 310--316.Google ScholarGoogle Scholar
  7. Charles Darwin and Phillip Prodger. 1998. The expression of the emotions in man and animals. Oxford University Press, USA.Google ScholarGoogle Scholar
  8. Arjun D'Cunha, Abhay Gupta, Kamal Awasthi, and Vineeth Balasubramanian. 2016. Daisee: Towards user engagement recognition in the wild. arXiv preprint arXiv:1609.01885 (2016).Google ScholarGoogle Scholar
  9. Abhinav Dhall, Amanjot Kaur, Roland Goecke, and Tom Gedeon. 2018. EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction. ACM ICMI (in press) (2018). Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Maria Frank, Ghassem Tofighi, Haisong Gu, and Renate Fruchter. 2016. Engagement detection in meetings. arXiv preprint arXiv:1608.08711 (2016).Google ScholarGoogle Scholar
  11. Daniel Gatica-Perez, L. McCowan, Dong Zhang, and Samy Bengio. 2005. Detecting group interest-level in meetings. In Acoustics, Speech, and Signal Processing, 2005. Proceedings. (ICASSP'05). IEEE International Conference on, Vol. Vol. 1. IEEE, I--489.Google ScholarGoogle Scholar
  12. Amanjot Kaur, Aamir Mustafa, Love Mehta, and Abhinav Dhall. 2018. Prediction and Localization of Student Engagement in the Wild. arXiv preprint arXiv:1804.00858 (2018).Google ScholarGoogle Scholar
  13. Reed W. Larson and Maryse H. Richards. 1991. Boredom in the middle school years: Blaming schools versus blaming students. American journal of education Vol. 99, 4 (1991), 418--443.Google ScholarGoogle Scholar
  14. Daniel F. O. Onah, Jane Sinclair, and Russell Boyatt. 2014. Dropout rates of massive open online courses: behavioural patterns. EDULEARN14 proceedings (2014), 5825--5834.Google ScholarGoogle Scholar
  15. Jinxian Qin, Yaqian Zhou, Hong Lu, and Heqing Ya. 2015. Teaching Video Analytics Based on Student Spatial and Temporal Behavior Mining Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. ACM, 635--642. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mirko Raca. 2015. Camera-based estimation of student's attention in class. (2015).Google ScholarGoogle Scholar
  17. Prajit Ramachandran, Barret Zoph, and Quoc V Le. {n.d.}. Searching for activation functions. ({n.d.}).Google ScholarGoogle Scholar
  18. Colin Lea Michael D. Flynn René and Vidal Austin Reiter Gregory D. Hager. 2017. Temporal convolutional networks for action segmentation and detection IEEE International Conference on Computer Vision (ICCV).Google ScholarGoogle Scholar
  19. Hanan Salam and Mohamed Chetouani. 2015. Engagement detection based on mutli-party cues for human robot interaction Affective Computing and Intelligent Interaction (ACII), 2015 International Conference on. IEEE, 341--347. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Gale M. Sinatra, Benjamin C. Heddy, and Doug Lombardi. 2015. The challenges of defining and measuring student engagement in science. (2015).Google ScholarGoogle Scholar
  21. Chinchu Thomas and Dinesh Babu Jayagopi. 2017. Predicting student engagement in classrooms using facial behavioral cues Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education. ACM, 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Ghassem Tofighi, Haisong Gu, and Kaamraan Raahemifar. 2016. Vision-based engagement detection in Virtual Reality Digital Media Industry & Academic Forum (DMIAF). IEEE, 202--206.Google ScholarGoogle Scholar
  23. Jacob Whitehill, Zewelanji Serpell, Yi-Ching Lin, Aysha Foster, and Javier R. Movellan. 2014. The faces of engagement: Automatic recognition of student engagementfrom facial expressions. IEEE Transactions on Affective Computing Vol. 5, 1 (2014), 86--98.Google ScholarGoogle ScholarCross RefCross Ref
  24. Erroll et al. Wood. 2015. Rendering of eyes for eye-shape registration and gaze estimation Proceedings of the IEEE International Conference on Computer Vision. 3756--3764. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Predicting Engagement Intensity in the Wild Using Temporal Convolutional Network

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Other conferences
          ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction
          October 2018
          687 pages
          ISBN:9781450356923
          DOI:10.1145/3242969

          Copyright © 2018 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 2 October 2018

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          ICMI '18 Paper Acceptance Rate63of149submissions,42%Overall Acceptance Rate453of1,080submissions,42%

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader