Abstract
The integration of robots into various industries, including manufacturing, has introduced new challenges in achieving efficient human-robot collaboration. A crucial aspect of successful collaboration is the ability of robots to understand and respond to human emotions. In the context of human-robot collaboration in manufacturing, accurately predicting human emotions is essential for enhancing efficiency and safety. This paper presents a setup for human emotion detection, focusing on facial emotion recognition. The proposed model and descriptive summary involve the utilising state-of-the-art algorithms such as AlexNet, HaarCascade (HCC), MTCNN (Multi-Task Cascaded Convolutional Neural Networks), and SVM (Support Vector Machine), applied to datasets like CK+, JAFFE, and AffectNet. The performance of each facial recognition model is evaluated in real-time scenarios, resulting in significant progress with an accuracy improvement from 40% to 78.1%. These results demonstrate the effectiveness of the approach in enabling adaptive robot control based on human emotions and enhancing collaboration quality. This research uniquely integrates facial emotion recognition and robot control to enable adaptive responses during human-robot collaboration in manufacturing settings. By understanding and responding to human emotions, robots can improve their interactions with humans, leading to increased productivity and improved overall collaboration efficiency (If EquinOCS, our proceedings submission system, is used, then the disclaimer can be provided directly in the system).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Eyam, A.T., Mohammed, W.M., Lastra, J.L.M.: Emotion-driven analysis and control of human-robot interactions in collaborative applications. Sensors 21, 4626 (2021).https://doi.org/10.3390/S21144626
Heredia, J., et al.: Adaptive multimodal emotion detection architecture for social robots. IEEE Access 10, 20727–20744 (2022). https://doi.org/10.1109/ACCESS.2022.3149214
Rawal, N., Stock-Homburg, R.M.: Facial emotion expressions in human-robot interaction: a survey. Int. J. Soc. Robot. 14, 1583–1604 (2022). https://doi.org/10.1007/S12369-022-00867-0/TABLES/5
Spezialetti, M., Placidi, G., Rossi, S.: Emotion recognition for human-robot interaction: recent advances and future perspectives. Front. Robot. AI 7, 532279 (2020). https://doi.org/10.3389/FROBT.2020.532279/BIBTEX
Khan, F., Asif, S., Webb, P.: Communication components for human intention prediction – a survey. Hum. Aspects Adv. Manuf. 80 (2023). https://doi.org/10.54941/AHFE1003504
Ali, M.F., Khatun, M.: Facial Emotion Detection Using Neural Network Low cost Education System View project BD classic movie restoration Project View project (2020)
Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: review of sensors and methods. Sensors 20, 592 (2020).https://doi.org/10.3390/S20030592
Cai, Y., Li, X., Li, J.: Emotion recognition using different sensors, emotion models, methods and datasets: a comprehensive review. Sensors 23, 2455 (2023).https://doi.org/10.3390/S23052455
Nguyen, B.T., Trinh, M.H., Phan, T.V., Nguyen, H.D.: An efficient real-time emotion detection using camera and facial landmarks. In: 7th International Conference on Information Science and Technology, ICIST 2017 – Proceedings, pp. 251–255 (2017). https://doi.org/10.1109/ICIST.2017.7926765
Mollahosseini, A., et al.: IEEE TRANSACTIONS ON AFFECTIVE COMPUTING AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Lucey, P., et al.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, pp. 94–101 (2010). https://doi.org/10.1109/CVPRW.2010.5543262
Lyons, M., Kamachi, M., Gyoba, J.: The Japanese Female Facial Expression (JAFFE) Dataset (1998). https://doi.org/10.5281/ZENODO.3451524
Krizhevsky, A., Inc, G.: One weird trick for parallelizing convolutional neural networks (2014)
Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests
Valagkouti, I.A., et al.: Emotion recognition in human–robot interaction using the NAO robot. Computers 11, 72 (2022). https://doi.org/10.3390/COMPUTERS11050072
Al-Atroshi, S.J.A., Ali, A.M.: Improving facial expression recognition using HOG with SVM and modified datasets classified by Alexnet. Traitement du Signal 40, 1611–1619 (2023). https://doi.org/10.18280/TS.400429
Minaee, S., Minaei, M., Abdolrashidi, A.: Deep-emotion: facial expression recognition using attentional convolutional network. Sensors 21 (2019). https://doi.org/10.3390/s21093046
Acknowledgment
This work was supported by EPSRC-funded Made Smarter Innovation - Research Centre for Smart, Collaborative Industrial Robotics project (EP/V062158/1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Khan, F., Asif, S., Webb, P. (2025). Human Facial Emotion Recognition for Adaptive Human Robot Collaboration in Manufacturing. In: Huda, M.N., Wang, M., Kalganova, T. (eds) Towards Autonomous Robotic Systems. TAROS 2024. Lecture Notes in Computer Science(), vol 15052. Springer, Cham. https://doi.org/10.1007/978-3-031-72062-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-72062-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72061-1
Online ISBN: 978-3-031-72062-8
eBook Packages: Computer ScienceComputer Science (R0)