Abstract
When micro-expressions are mixed with normal or macro-expressions, it becomes increasingly challenging to spot them in long videos. Aiming at the specific time prior of Micro-expression (ME)s, a ME spotting network called AEM-Net (Adaptive Enhanced ME Detection Network) is proposed. The network improves the spotting performance in the following four aspects. First, the multi-stage channel feature extraction module is proposed to extract feature information of different depths. Then, an attention spatial-temporal module was used to obtain salient and discriminative micro-expression segments while suppressing the generation of excessively long or short suggestions. Thirdly, a ME-NMS (Non-Maximum Suppression) network is developed to reduce redundancy and decision errors. Finally, two spotting mechanisms named anchor_based and anchor_free are combined in our method. Extensive experiments have done on prevalent \(\mathrm CAS(ME)^2\) and the SAMM-Long ME databases to evaluate the spotting performance. The results show that the AEM-Net achieves an impressive performance, which outperforms other state-of-the-art methods.
Z. Xie—This work is supported by the National Nature Science Foundation of China (No. 61861020), the Jiangxi Province Graduate Innovation Special Fund Project (No. YC2021-X06).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Porter, S., Brinke, L.: Reading between the lies: identifying concealed and falsified emotions in universal facial expressions. Psychol. Sci. 19(5), 508–514 (2008)
Ekman, P., Friesen, W. V.: Nonverbal leakage and clues to deception. Psychiatry 32(1), 88–106 (1969)
Porter, S., Brinke, L.: Reading between the lies identifying concealed and falsified emotions in universal facial expressions. Psychol. Sci. 19(5), 508–514 (2008)
Zhang, Z., Chen, T. H., Liu, G., Fu, X.: SMEconvnet: a convolutional neural network for spotting spontaneous facial micro-expression from long videos. IEEE Access 6(71), 143–171 (2018)
Antti, M., Guoying, Z., Matti, P.: Spotting rapid facial movements from videos using appearance-based feature difference analysis. In: International Conference on Pattern Recognition, pp. 1722–1727 (2014)
Adrian, D.K., Moi, Y.H., Cliff, L.: Micro-facial movement detection using individualised baselines and histogram based descriptors. In: International Conference on Systems, Man, and Cybernetics, pp. 1864–1869 (2015)
Adrian, D., Walied, M., Cliff, L., Choon, N.C., Moi, Y.H.: Objective micro-facial movement detection using FACS-based regions and baseline evaluation. In: International Conference on Automatic Face and Gesture Recognition (FG), pp. 642–649 (2018)
Devangini, P., Guoying, Z., Matti, P.: Spatiotemporal integration of optical flow vectors for micro-expression detection. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 369–380 (2015)
Thuong, T.K., Xiaopeng, H., Guoying, Z.: Sliding window based micro-expression spotting: a benchmark. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 542–553 (2017)
Sujing, W., Shuhuang, W., Xingsheng, Q., Jingxiu, L., Xiaolan, F.: A main directional maximal difference analysis for spotting facial movements from long-term videos. Neurocomputing 382–389 (2017)
Genbing, L., See, J., Laikuan, W.: Shallow optical flow three-stream CNN for macro-and micro-expression spotting from long videos. In: ICIP, pp. 2643–2647 (2021)
Wangwang, Y., Jingwen, J., Yongjie, L.: LSSNET: a two-stream convolutional neural network for spotting macro-and micro-expression in long videos. In: ACM Conference on Multimedia, pp. 4745–4749 (2021)
Xiaolong, W., Girshick, R., Gupta, A., Kaiming, H.: Non-local neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)
Li, J., Soladie, C., Seguier, R., Wang, S.-J., Yap, M.H.: Spotting micro-expressions on long videos sequences. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 1–5 (2019)
Qu, F., Wang, S.J., Yan, W. J., Li, H., Wu, S., Fu, X.: CAS(ME) 2: a database for spontaneous macro-expression and micro-expression spotting and recognition. IEEE Trans. Affect. Comput. 9(4), 424–436 (2017)
Yap, C., Kendrick, C., Yap, M.: SAMM long videos: a spontaneous facial micro-and macro-expressions dataset. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 194–199 (2020)
Yap, C., Yap, M., Davison, A., Cunningham, R.: 3D-CNN for facial micro- and macro-expression spotting on long video sequences using temporal oriented reference frame. arXiv:2105.06340 (2021)
Wang, S., He, Y., Li, J., Fu, X.: MESNet: a convolutional neural network for spotting multi-scale micro-expression intervals in long videos. IEEE Trans. Image Process. 3956–3969 (2021)
Zhang, L., et al.: Spatio-temporal fusion for macro-and micro-expression spotting in long video sequences. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 245–252 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Z., Cheng, S., Liu, X., Fan, J. (2022). Adaptive Enhanced Micro-expression Spotting Network Based on Multi-stage Features Extraction. In: Deng, W., et al. Biometric Recognition. CCBR 2022. Lecture Notes in Computer Science, vol 13628. Springer, Cham. https://doi.org/10.1007/978-3-031-20233-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-20233-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20232-2
Online ISBN: 978-3-031-20233-9
eBook Packages: Computer ScienceComputer Science (R0)