Abhinav Dhall
Roland Goecke
ABSTRACT
This paper describes the 9th Emotion Recognition in the Wild (EmotiW) challenge, which is being run as a grand challenge at the 25th ACM International Conference on Multimodal Interaction 2023. EmotiW challenge focuses on affect related benchmarking tasks and comprises of two sub-challenges: a) User Engagement Prediction in the Wild, and b) Audio-Visual Group-based Emotion Recognition. The purpose of this challenge is to provide a common platform for researchers from diverse domains. The objective is to promote the development and assessment of methods, which can predict engagement levels and/or identify perceived emotional well-being of a group of individuals in real-world circumstances. We describe the datasets, the challenge protocols and the accompanying sub-challenge.
- Asad Abbas and Stephan K. Chalup. 2017. Group Emotion Recognition in the Wild by Combining Deep Neural Networks for Facial Expression Classification and Scene-Context Analysis. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI ’17). Association for Computing Machinery, New York, NY, USA, 561–568. https://doi.org/10.1145/3136755.3143010Google Scholar
Digital Library
- Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall, Jianfei Cai, Hamid Rezatofighi, Reza Haffari, and Munawar Hayat. 2023. MARLIN: Masked Autoencoder for Facial Video Representation LearnINg. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1493–1504.Google ScholarCross Ref
- Abhinav Dhall. 2019. EmotiW 2019: Automatic Emotion, Engagement and Cohesion Prediction Tasks. In 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI ’19). Association for Computing Machinery, New York, NY, USA, 546–550. https://doi.org/10.1145/3340555.3355710Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, and Tom Gedeon. 2015. Automatic group happiness intensity analysis. IEEE Transactions on Affective Computing 6, 1 (2015), 13–26.Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, Shreya Ghosh, Jyoti Joshi, Jesse Hoey, and Tom Gedeon. 2017. From Individual to Group-Level Emotion Recognition: EmotiW 5.0. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI ’17). Association for Computing Machinery, New York, NY, USA, 524–528. https://doi.org/10.1145/3136755.3143004Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, Jyoti Joshi, Jesse Hoey, and Tom Gedeon. 2016. EmotiW 2016: Video and Group-Level Emotion Recognition Challenges. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (Tokyo, Japan) (ICMI ’16). Association for Computing Machinery, New York, NY, USA, 427–432. https://doi.org/10.1145/2993148.2997638Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, Jyoti Joshi, Karan Sikka, and Tom Gedeon. 2014. Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol. In Proceedings of the 16th International Conference on Multimodal Interaction (Istanbul, Turkey) (ICMI ’14). Association for Computing Machinery, New York, NY, USA, 461–466. https://doi.org/10.1145/2663204.2666275Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, Jyoti Joshi, Michael Wagner, and Tom Gedeon. 2013. Emotion Recognition in the Wild Challenge 2013. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (Sydney, Australia) (ICMI ’13). Association for Computing Machinery, New York, NY, USA, 509–516. https://doi.org/10.1145/2522848.2531739Google ScholarDigital Library
- Abhinav Dhall, Roland Goecke, Simon Lucey, Tom Gedeon, 2012. Collecting large, richly annotated facial-expression databases from movies. IEEE multimedia 19, 3 (2012), 34.Google ScholarDigital Library
- Abhinav Dhall, Jyoti Joshi, Karan Sikka, Roland Goecke, and Nicu Sebe. 2015. The more the merrier: Analysing the affect of a group of people in images. In 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), Vol. 1. IEEE, 1–8.Google ScholarCross Ref
- Abhinav Dhall, Amanjot Kaur, Roland Goecke, and Tom Gedeon. 2018. EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI ’18). Association for Computing Machinery, New York, NY, USA, 653–656. https://doi.org/10.1145/3242969.3264993Google ScholarDigital Library
- Abhinav Dhall, O.V. Ramana Murthy, Roland Goecke, Jyoti Joshi, and Tom Gedeon. 2015. Video and Image Based Emotion Recognition Challenges in the Wild: EmotiW 2015. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (Seattle, Washington, USA) (ICMI ’15). Association for Computing Machinery, New York, NY, USA, 423–426. https://doi.org/10.1145/2818346.2829994Google ScholarDigital Library
- Abhinav Dhall, Garima Sharma, Roland Goecke, and Tom Gedeon. 2020. EmotiW 2020: Driver Gaze, Group Emotion, Student Engagement and Physiological Signal Based Challenges. In Proceedings of the 2020 International Conference on Multimodal Interaction (Virtual Event, Netherlands) (ICMI ’20). Association for Computing Machinery, New York, NY, USA, 784–789. https://doi.org/10.1145/3382507.3417973Google ScholarDigital Library
- Florian Eyben, Felix Weninger, Florian Gross, and Björn Schuller. 2013. Recent Developments in OpenSMILE, the Munich Open-Source Multimedia Feature Extractor. In Proceedings of the 21st ACM International Conference on Multimedia (Barcelona, Spain) (MM ’13). Association for Computing Machinery, New York, NY, USA, 835–838. https://doi.org/10.1145/2502081.2502224Google ScholarDigital Library
- Shreya Ghosh, Abhinav Dhall, and Nicu Sebe. 2018. Automatic Group Affect Analysis in Images via Visual Attribute and Feature Networks. In 2018 25th IEEE International Conference on Image Processing (ICIP). 1967–1971. https://doi.org/10.1109/ICIP.2018.8451242Google ScholarCross Ref
- S. Ghosh, A. Dhall, G. Sharma, S. Gupta, and N. Sebe. 2021. Speak2Label: Using Domain Knowledge for Creating a Large Scale Driver Gaze Zone Estimation Dataset. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). IEEE Computer Society, Los Alamitos, CA, USA, 2896–2905. https://doi.org/10.1109/ICCVW54120.2021.00324Google ScholarCross Ref
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. 9, 8 (nov 1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735Google ScholarDigital Library
- Amanjot Kaur, Aamir Mustafa, Love Mehta, and Abhinav Dhall. 2018. Prediction and Localization of Student Engagement in the Wild. In 2018 Digital Image Computing: Techniques and Applications (DICTA). 1–8. https://doi.org/10.1109/DICTA.2018.8615851Google ScholarCross Ref
- Amanjot Kaur, Aamir Mustafa, Love Mehta, and Abhinav Dhall. 2018. Prediction and localization of student engagement in the wild. In 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 1–8.Google Scholar
- Jingting Li, Moi Hoon Yap, Wen-Huang Cheng, John See, Xiaopeng Hong, Xiaobai Li, Su-Jing Wang, Adrian K. Davison, Yante Li, and Zizhao Dong. 2022. MEGC2022: ACM Multimedia 2022 Micro-Expression Grand Challenge. In Proceedings of the 30th ACM International Conference on Multimedia (Lisboa, Portugal) (MM ’22). Association for Computing Machinery, New York, NY, USA, 7170–7174. https://doi.org/10.1145/3503161.3551601Google ScholarDigital Library
- Hamed Monkaresi, Nigel Bosch, Rafael A. Calvo, and Sidney K. D’Mello. 2017. Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate. IEEE Transactions on Affective Computing 8, 1 (2017), 15–28. https://doi.org/10.1109/TAFFC.2016.2515084Google ScholarDigital Library
- Alexandr Rassadin, Alexey Gruzdev, and Andrey Savchenko. 2017. Group-Level Emotion Recognition Using Transfer Learning from Face Identification. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI ’17). Association for Computing Machinery, New York, NY, USA, 544–548. https://doi.org/10.1145/3136755.3143007Google ScholarDigital Library
- Fabien Ringeval, Björn Schuller, Michel Valstar, Roddy Cowie, Heysem Kaya, Maximilian Schmitt, Shahin Amiriparian, Nicholas Cummins, Denis Lalanne, Adrien Michaud, Elvan Ciftçi, Hüseyin Güleç, Albert Ali Salah, and Maja Pantic. 2018. AVEC 2018 Workshop and Challenge: Bipolar Disorder and Cross-Cultural Affect Recognition. In Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop (Seoul, Republic of Korea) (AVEC’18). Association for Computing Machinery, New York, NY, USA, 3–13. https://doi.org/10.1145/3266302.3266316Google ScholarDigital Library
- Garima Sharma, Shreya Ghosh, and Abhinav Dhall. 2019. Automatic Group Level Affect and Cohesion Prediction in Videos. In 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). 161–167. https://doi.org/10.1109/ACIIW.2019.8925231Google ScholarCross Ref
- Monisha Singh, Ximi Hoque, Donghuo Zeng, Yanan Wang, Kazushi Ikeda, and Abhinav Dhall. 2023. Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines. arXiv preprint arXiv:2302.00431 (2023).Google Scholar
- Qinglan Wei, Yijia Zhao, Qihua Xu, Liandong Li, Jun He, Lejun Yu, and Bo Sun. 2017. A New Deep-Learning Framework for Group Emotion Recognition. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI ’17). Association for Computing Machinery, New York, NY, USA, 587–592. https://doi.org/10.1145/3136755.3143014Google ScholarDigital Library
- Jacob Whitehill, Zewelanji Serpell, Yi-Ching Lin, Aysha Foster, and Javier R. Movellan. 2014. The Faces of Engagement: Automatic Recognition of Student Engagement from Facial Expressions. IEEE Transactions on Affective Computing 5, 1 (2014), 86–98. https://doi.org/10.1109/TAFFC.2014.2316163Google ScholarCross Ref
Index Terms
- EmotiW 2023: Emotion Recognition in the Wild Challenge
Recommendations
EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction
ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal InteractionThis paper details the sixth Emotion Recognition in the Wild (EmotiW) challenge. EmotiW 2018 is a grand challenge in the ACM International Conference on Multimodal Interaction 2018, Colarado, USA. The challenge aims at providing a common platform to ...
EmotiW 2020: Driver Gaze, Group Emotion, Student Engagement and Physiological Signal based Challenges
ICMI '20: Proceedings of the 2020 International Conference on Multimodal InteractionThis paper introduces the Eighth Emotion Recognition in the Wild (EmotiW) challenge. EmotiW is a benchmarking effort run as a grand challenge of the 22nd ACM International Conference on Multimodal Interaction 2020. It comprises of four tasks related to ...
Emotion recognition in the wild challenge (EmotiW) challenge and workshop summary
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interactionThe Emotion Recognition In The Wild Challenge and Workshop (EmotiW) 2013 Grand Challenge consists of an audio-video based emotion classification challenge, which mimics real-world conditions. In total, 27 teams participated in the challenge. The ...
Comments