Abstract
Various Facial Expression Recognition (FER) systems have been studied in the field of computer vision and machine learning to encode expression information from facial representations. In this research paper, a facial emotion recognition system is proposed, addressing automatic face detection and facial expression recognition using 1) Residual Neural Network (RESNET) and 2) Combined Residual Neural Network + Long Short-Term Memory (Combined RESNET+LSTM). The architectures of RESNET and Combined RESNET+LSTM are inspired by the human retina structure and human primary visual cortex structure. The proposed architectures are compared with each other. They are tested using a challenging public database.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fer dataset (2013). https://www.kaggle.com/msambare/fer2013/
Amidi, A.S.A.: Vip cheatsheet: Reccurent neural network (2019). https://stanford.edu/ shervine
Mollahosseini, A.D., Chan, M.M.: Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision WACV, pp. 1–10 (2016)
Kim, B.-K., Dong, S.-Y., Roh, J., Kim, G., Lee, S.-Y.: Fusing aligned and non-aligned face information for automatic affect recognition in the wild: a deep learning approach. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops CVPRW (2016)
Benamara, N.K., et al.: Real-time emotional recognition for sociable robotics based on deep neural networks ensemble. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo Moreo, J., Adeli, H. (eds.) IWINAC 2019. LNCS, vol. 11486, pp. 171–180. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19591-5_18
Benamara, N.: Real-time facial expression recognition using smoothed deep neural network ensemble. Integrat. Comput.-Aid. Eng. 28(1), 97–111 (2021)
Pramerdorfer, C., Kampel, M.: Facial expression recognition using convolutional neural networks: State of the art. ArXiv:1612.02903 (2016)
Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: International Conference on Multimodal Interaction ICMI, pp. 503–510 (2016)
Redmon, J., Farhadi, A.: Yolo9000: Better, faster, stronger. In: IEEE Conference on Computer Vision and Pattern Recognition CVPR 11486 (2017)
Ferrandez, J.M., et al.: Brain plasticity: feasibility of a cortical visual prosthesis for the blind. In: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society vol. 3, pp. 2027–2030 (2003)
Ferrandez, J.M., Liaño, E., Bonomini, P., Martinez, J.J., Toledo, J., Fernandez, E.: A customizable multi-channel stimulator for cortical neuroprosthesis. In: 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4607–4710 (2007)
Multimedia Laboratory, D.o.I.E.T.C.U.o.H.K.: Wider face dataset. http://shuoyang1213.me/WIDERFACE/ (2019)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 11486 (2001)
Li, S.W.D.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. 1(1) (2020)
Devries, T., Biswaranjan, K.T., Graham W.: Multi-task learning of facial landmarks and expression. In: Canadian Conference on Computer and Robot Vision (2014)
Guo, Y., Tao, D., Yu, J., Xiong, H., Li, Y., Tao, D.: Deep neural networks with relativity learning for facial expression recognition. In: IEEE International Conference on Multimedia Expo Workshops (ICMEW)) (2016)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(22) (1998)
Yang, Y.: Deep learning using linear support vector machines. ArXiv:1306.0239 (2013)
Lian, Z., Li, Y., Tao, J.-H., Huang, J., Niu, M.-Y.: Expression analysis based on face regions in real-world conditions. Int. J. Autom. Comput. 17(1), 96–107 (2019). https://doi.org/10.1007/s11633-019-1176-9
Acknowledgements
This project has received funding by grant RTI2018-098969-B-100 from the Spanish Ministerio de Ciencia Innovación y Universidades and by grant PROMETEO/2019/119 from the Generalitat Valenciana (Spain), and by Grant PID2020-115220RB-C22 funded by MCIN/AEI/ 10.13039/501100011033 and, as appropriate, by “ERDF A way of making Europe”, by the “European Union” or by the “European Union NextGenerationEU/PRTR”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Huq, M., Garrigos, J., Martinez, J.J., Ferrandez, J., Fernández, E. (2022). Application of RESNET and Combined RESNET+LSTM Network for Retina Inspired Emotional Face Recognition System. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Artificial Intelligence in Neuroscience: Affective Analysis and Health Applications. IWINAC 2022. Lecture Notes in Computer Science, vol 13258. Springer, Cham. https://doi.org/10.1007/978-3-031-06242-1_63
Download citation
DOI: https://doi.org/10.1007/978-3-031-06242-1_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06241-4
Online ISBN: 978-3-031-06242-1
eBook Packages: Computer ScienceComputer Science (R0)