Abstract
Emotion recognition from facial expressions is pivotal in enhancing human-computer interaction (HCI). Spanning across diverse applications such as virtual assistants, mental health support, and personalized content recommendations, it promises to revolutionize how we interact with technology. This study explores the effectiveness of the Vision Transformer (ViT) architecture within the context of emotion classification, leveraging a rich dataset. Our research methodology is characterized by meticulous preprocessing, extensive data augmentation, and fine-tuning of the ViT model. For experiment purposes, we use the Emotion Detection FER-2013 dataset. Rigorous evaluation metrics are meticulously employed to gauge the model’s performance. The research underscores the potential for enhancing user experiences, facilitating mental health monitoring, and navigating the ethical considerations inherent in emotion-aware technologies. Our model achieved a testing accuracy of 70%. As we chart new horizons in HCI, future endeavors should focus on fine-tuning model accuracy across various emotion categories and navigating the complexities of real-world deployment challenges.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Han, K., et al.: A survey on vision transformer. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 87–110 (2022)
Barling, J., Slater, F., Kelloway, E.K.: Transformational leadership and emotional intelligence: an exploratory study. Leadersh. Org. Dev. J. 21(3), 157–161 (2000)
Lee, C.-C., Shih, C.-Y., Lai, W.-P., Lin, P.-C.: An improved boosting algorithm and its application to facial emotion recognition. J. Ambient. Intell. Humaniz. Comput. 3, 11–17 (2012)
Feng, X., Pietikainen, M., Hadid, A.: Facial expression recognition based on local binary patterns. Pattern Recognit Image Anal. 17, 592–598 (2007)
Chang, C.-Y., Huang, Y.-C.: Personalized facial expression recognition in indoor environments. In: Proceedings of the 2010 International Joint Conference on Neural Networks (IJCNN), Barcelona, Spain, pp. 1–8 (2010)
Peddoju, S.K., Upadhyay, H., Soni, J., Prabakar, N.: Natural language processing-based anomalous system call sequences detection with virtual memory introspection. Int. J. Adv. Comput. Sci. Appl. 11(5) (2020)
Soni, J., Prabakar, N., Upadhyay, H.: Deep learning approach to detect malicious attacks at system level: poster. In: Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, pp. 314–315 (2019)
Xiao, X.Q., Wei, J.: Application of wavelet energy feature in facial expression recognition. In: Proceedings of the 2007 International Workshop on Anti-Counterfeiting, Security, and Identification (ASID), Xiamen, China, pp. 169–174 (2007)
Pranav, E., Kamal, S., Chandran, C.S., Supriya, M.: Facial emotion recognition using deep convolutional neural network. In: Proceedings of the 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, pp. 317–320 (2020)
Pons, G., Masip, D.: Supervised committee of convolutional neural networks in automated facial expression analysis. IEEE Trans. Affect. Comput. 9, 343–350 (2018)
Kosti, R., Alvarez, J.M., Recasens, A., Lapedriza, A.: Context-based emotion recognition using EMOTIC dataset. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2755–2766 (2020)
Wei, M., Zheng, W., Zong, Y., Jiang, X., Lu, C., Liu, J.: A novel micro-expression recognition approach using attention-based magnification-adaptive networks. In: Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Singapore, pp. 2420–2424 (2022)
Li, J., et al.: Facial expression recognition by transfer learning for small datasets. In: Yang, C.-N., Peng, S.-L., Jain, L.C. (eds.) SICBS 2018. AISC, vol. 895, pp. 756–770. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-16946-6_62
Dutta, S., Ganapathy, S.: Multimodal transformer with learnable frontend and self attention for emotion recognition. In: Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Singapore, pp. 6917–6921 (2022)
Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., Wang, Y.: Transformer in transformer. Adv. Neural. Inf. Process. Syst. 34, 15908–15919 (2021)
Shaw, P., Uszkoreit, J., Vaswani, A.: Self-attention with relative position representations (2018). arXiv preprint arXiv:1803.02155
Wolf, T., et al.: Huggingface’s transformers: State-of-the-art natural language processing (2019). arXiv preprint arXiv:1910.03771
Paszke, A., et al.: Automatic differentiation in pytorch (2017)
Saravanan, A, Perichetla, G, Gayathri, D.K.: Facial emotion recognition using convolutional neural networks (2019). arXiv preprint arXiv:1910.05602
Gan, Y.: Facial expression recognition using convolutional neural network. In: Proceedings of the 2nd International Conference on Vision, Image and Signal Processing, pp. 1–5 (2018)
Wang, Y., Li, Y., Song, Y., Rong, X.: Facial expression recognition based on auxiliary models. Algorithms 12(11), 227 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Soni, J., Prabakar, N., Upadhyay, H. (2024). Vision Transformer-Based Emotion Detection in HCI for Enhanced Interaction. In: Choi, B.J., Singh, D., Tiwary, U.S., Chung, WY. (eds) Intelligent Human Computer Interaction. IHCI 2023. Lecture Notes in Computer Science, vol 14531. Springer, Cham. https://doi.org/10.1007/978-3-031-53827-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-53827-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-53826-1
Online ISBN: 978-3-031-53827-8
eBook Packages: Computer ScienceComputer Science (R0)