Abstract
Micro-expression (ME) is a kind of facial muscle movement spontaneously, which can reflect people’s real emotions and be widely used in psychological treatment, suspect interrogation and other fields. However, the intensity and duration of ME pose enormous challenges to robust micro-expression recognition (MER). In this work, a capsule network based on local optical flow features (LOFCAP) is proposed to explore efficient MER. In the MER based on LOFCAP, the input ME images are divided into small blocks with the same size. Then, each one is sent to residual network (ResNet) convolutional layer for feature extraction. Finally, the fused features are sent into the capsule network for final classification. SMIC, SAMM and CASME II databases are used to validate experimental results. Furthermore, unweighted F1 score (UF1) and unweighted average recall (UAR) are used as evaluation metrics. Especially, when the image is divided into 9 blocks, UF1 and UAR reached 0.8104 and 0.8403, respectively. Experimental results from full micro-expression datasets show that our LOFCAP model can effectively represent the local features on ME and the overall performance is successful than the state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ekman, P., Friesen, W.V.: Nonverbal leakage and clues to deception. Psychiatry 32(1), 88–106 (1969)
Yan, W.J., Wu, Q., Liang, J., et al.: How fast are the leaked facial expressions. J. Nonverbal Behav. 37(4), 217–230 (2013)
Song, B., Zong, Y., Li, K., et al.: Cross-database micro-expression recognition based on a dual-stream convolutional neural network. IEEE Access 10, 66227–66237 (2022)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
Liong, S.T., See, J., Wong, K.S., et al.: Less is more: Micro-expression recognition from video using apex frame. Sign. Process. Image Commun. 62, 82–92 (2018)
Zong, Y., Huang, X., Zheng, W., et al.: Learning from hierarchical spatiotemporal descriptors for micro-expression recognition. IEEE Trans. Multimed. 20(11), 3160–3172 (2018)
Wei, J., Lu, G., Yan, J.: Learning two groups of discriminative features for micro-expression recognition. Neurocomputing 479, 22–36 (2022)
Van Quang, N., Chun, J., Tokuyama, T.: CapsuleNet for micro-expression recognition. In: IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1–7 (2019)
Xie, Z., Shi, L., Cheng, S., et al.: Micro-expression recognition based on deep capsule adversarial domain adaptation network. J. Electron. Imaging 31(1), 013021 (2022)
Chen, B., Liu, K., Xu, Y., et al.: Block division convolutional network with implicit deep features augmentation for micro-expression recognition. IEEE Trans. Multimed. 25, 1345–1358 (2022)
Lei, L., Chen, T., Li, S., et al.: Micro-expression recognition based on facial graph representation learning and facial action unit fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1571–1580 (2021)
Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. In: Bigun, J., Gustavsson, T. (eds.) Image Analysis: 13th Scandinavian Conference, SCIA 2003. Lecture Notes in Computer Science, vol. 2749, pp. 363–370. Springer, Berlin (2003). https://doi.org/10.1007/3-540-45103-X_50
Sabour, S., Frosst, N., Hinton, GE.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, vol, 30 (2017)
Li, X., Pfister, T., Huang, X., et al.: A spontaneous micro-expression database: inducement, collection and baseline. In: IEEE International Conference and Workshops on Automatic face and gesture recognition. 1–6 (2013)
Davison, A.K., Lansley, C., Costen, N., et al.: SAMM: a spontaneous micro-facial movement dataset. IEEE Trans. Affect. Comput. 9(1), 116–129 (2016)
Yan, W.J., Li, X., Wang, S.J., et al.: CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1), e86041 (2014)
Acknowledgements
This paper is supported by the Natural Science Foundation of Jiangxi Province of China (No.20224ACB202011), the National Nature Science Foundation of China (No.61861020) and the Jiangxi Province Graduate Innovation Special Fund Project (No. YC2022-s790).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Z., Liu, X. (2023). Micro-expression Recognition Based on Local Optical Flow Capsule Network. In: Fujita, H., Wang, Y., Xiao, Y., Moonis, A. (eds) Advances and Trends in Artificial Intelligence. Theory and Applications. IEA/AIE 2023. Lecture Notes in Computer Science(), vol 13925. Springer, Cham. https://doi.org/10.1007/978-3-031-36819-6_35
Download citation
DOI: https://doi.org/10.1007/978-3-031-36819-6_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36818-9
Online ISBN: 978-3-031-36819-6
eBook Packages: Computer ScienceComputer Science (R0)