Abstract
This paper studies emotion detection using deep learning on the prevalent usage of face masks in the Covid-19 pandemic. Internet repository data Karolinska Directed Emotional Faces (KDEF) [1] was used as a base database, in which it was segmented into different portions of the face, such as forehead patch, eye patch, and skin patch to be representing segments of the face covered or exposed by the mask were transfer learned to an Inception v3 model. Results show that the full-face model had the highest accuracy 74.68% followed by the skin patch (area occluded by the mask) 65.09%. The models trained on full-face were then used to inference the different face segments/patches that showed poor inferencing results. However, certain emotions are more distinct around the eye region. Therefore, this paper concludes that upper segmented faces result in higher accuracy for training models over full faces, yet future research needs to be done on additional occlusion near the eye section.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lundqvist, D.E., Flykt, A., Öhman, A.: The karolinska directed emotional faces - KDEF, CD ROM from Department of Clinical Neuroscience. Psychology section, Karolinska Institutet (1998). ISBN 91–630–7164–9
Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. PR 36(1), 259–275 (2003). https://doi.org/10.1016/S0031-3203(02)00052-3
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)
Mehrabian, A.: Silent Messages - A Wealth of Information About Nonverbal Communication (Body Language). Silent Messages, Belmont (1981)
Grahlow, M., Rupp, C., Dernt, B.: The impact of face masks on emotion recognition performance and perception of threat (2021). https://doi.org/10.31234/osf.io/6msz8, Accessed 27 July 2021
Ranzato,M., Susskind, J., Mnih, V., Hinton, G.: On deep generative models with applications to recognition. In: CVPR 2011, pp. 2857–2864 (2011). https://doi.org/10.1109/CVPR.2011.5995710
Teoh, K.H., et al.: Face recognition and identification using deep learning approach. J. Phys. Conf. Ser. 1755(1), 012006 (2021). https://doi.org/10.1088/1742-6596/1755/1/012006
Li, Y., Guo, K., Lu, Y., Liu, L.: Cropping and attention based approach for masked face recognition. Appl. Intell. 51(5), 3012–3025 (2021). https://doi.org/10.1007/s10489-020-02100-9
Hariri, W.: Efficient masked face recognition method during the COVID-19 pandemic (2020). PREPRINT (Version 1) available at Research Square, https://doi.org/10.21203/rs.3.rs-39289/v1
Pinkney, J.: MTCNN face detection. https://github.com/matlab-deep-learning/mtcnn-face-detection/releases/tag/v1.2.4, GitHub. Accessed 26 July 2021
Pretrained Deep Neural Network. https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html#References, Accessed 25 July 2021
Inceptionv3. https://www.mathworks.com/help/deeplearning/ref/inceptionv3.html, Accessed 25 July 2021
Sajjanhar, A., Wu, Z., Wen, Q.: Deep learning models for facial expression recognition. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6 (2018). https://doi.org/10.1109/DICTA.2018.8615843
Salvador, R.C., Bandala, A.A., Javel, I.M., Bedruz, R.A.R., Dadios, E.P., Vicerra, R.R.P.: DeepTronic: an electronic device classification model using deep convolutional neural networks. In: 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology,Communication and Control, Environment and Management (HNICEM), pp. 1–5 (2018). https://doi.org/10.1109/HNICEM.2018.8666303
Zhang, L., Tjondronegoro, D.: Facial expression recognition using facial movement features. IEEE Trans. Affect. Comput. 2(4), 219–229 (2011). https://doi.org/10.1109/T-AFFC.2011.13
Acknowledgements
This research is supported by a TNB SEED grant managed by UNITEN R&D U-TD-TD-19–28.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Saravanan, P. et al. (2021). A Performance Study on Emotion Models Detection Accuracy in a Pandemic Environment. In: Badioze Zaman, H., et al. Advances in Visual Informatics. IVIC 2021. Lecture Notes in Computer Science(), vol 13051. Springer, Cham. https://doi.org/10.1007/978-3-030-90235-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-90235-3_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90234-6
Online ISBN: 978-3-030-90235-3
eBook Packages: Computer ScienceComputer Science (R0)