Cheng Liang
ABSTRACT
Micro-expression is the facial expression that reveals hidden true feelings. It is characterized by subtle movements and extremely short duration. Aiming at overcoming the shortcomings of existing algorithms, this paper proposes a micro-expression recognition method named DOF-CBAM-C3D. The method combines DOF (Dense Optical Flow) and deep learning methods, A simplified C3D (Convolutional 3D network) is used as backbone to extract the features from the temporal and spatial dimensions. The CBAM (Convolutional Block Attention Module) module is added to improve the network's perception on the importance of different channel and local area. DOF is extracted in the preprocessing stage to obtain the dynamic features in the time dimension, which compensates for the shortcomings of C3D using local aggregation mode to extract features of time dimension, while reducing the redundant information in the original images. Experiments on CAS(ME)3, CASMEII and SAMM datasets get UF1 of 80.32%, 82.14% and 81.07%, respectively, demonstrating the effectiveness of the proposed micro-expression recognition system.
- Ben, X, M Yang, Peng Zhang, 2014. Survey on automatic micro expression recognition methods. J. Comput.-Aided Design Comput. Graph. 26(9): p. 1385-1395.Google Scholar
- Ekman, Paul and Wallace V Friesen. 1969. Nonverbal leakage and clues to deception. Psychiatry. 32(1): p. 88-106.Google ScholarCross Ref
- Ben, Xianye, Yi Ren, Junping Zhang, 2021. Video-based facial micro-expression analysis: A survey of datasets, features and algorithms. IEEE transactions on pattern analysis and machine intelligence. 44(9): p. 5826-5846.Google Scholar
- Pan, Hang, Lun Xie, Zhiliang Wang, 2021. Review of micro-expression spotting and recognition in video sequences. Virtual Reality & Intelligent Hardware. 3(1): p. 1-17.Google ScholarCross Ref
- Cai, Linqin, Hao Li, Wei Dong, 2022. Micro-expression recognition using 3D DenseNet fused Squeeze-and-Excitation Networks. Applied Soft Computing. 119: p. 108594.Google ScholarDigital Library
- Liu, Kun-Hong, Qiu-Shi Jin, Huang-Chao Xu, 2021. Micro-expression recognition using advanced genetic algorithm. Signal Processing: Image Communication. 93: p. 116153.Google ScholarCross Ref
- Wang, Yi, Shixin Zheng, Xiao Sun, 2022. Micro-expression recognition with attention mechanism and region enhancement. Multimedia Systems: p. 1-9.Google Scholar
- Wang, Chongyang, Min Peng, Tao Bi, 2020. Micro-attention for micro-expression recognition. Neurocomputing. 410: p. 354-362.Google ScholarCross Ref
- Liu, Shuhua, Yushu Ren, Lanting Li, 2022. Micro-expression recognition based on SqueezeNet and C3D. Multimedia Systems: p. 1-10.Google Scholar
- Van Quang, Nguyen, Jinhee Chun, and Takeshi Tokuyama.2019 CapsuleNet for micro-expression recognition. in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). IEEE.Google ScholarDigital Library
- Xia, Zhaoqiang, Xiaopeng Hong, Xingyu Gao, 2019. Spatiotemporal recurrent convolutional networks for recognizing spontaneous micro-expressions. IEEE Transactions on Multimedia. 22(3): p. 626-640.Google ScholarCross Ref
- Ruhan, Liu and Xu Dan. 2019. Video amplification and deep learning in micro-expression recognition. Journal of Computer-Aided Design & Computer Graphics. 31(9): p. 1535-1541.Google Scholar
- Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.Google Scholar
- Pan, Hang, Lun Xie, and Zhiliang Wang. 2023. C3DBed: Facial micro-expression recognition with three-dimensional convolutional neural network embedding in transformer model. Engineering Applications of Artificial Intelligence. 123: p. 106258.Google ScholarDigital Library
- Nguyen, Xuan-Bac, Chi Nhan Duong, Xin Li, 2023 Micron-BERT: BERT-based Facial Micro-Expression Recognition. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Google Scholar
- Reddy, Sai Prasanna Teja, Surya Teja Karri, Shiv Ram Dubey, 2019 Spontaneous facial micro-expression recognition using 3D spatiotemporal convolutional neural networks. in 2019 International Joint Conference on Neural Networks (IJCNN). IEEE.Google ScholarCross Ref
- Tran, Du, Lubomir Bourdev, Rob Fergus, 2015 Learning spatiotemporal features with 3d convolutional networks. in Proceedings of the IEEE international conference on computer vision.Google ScholarDigital Library
- Xu, Feng, Junping Zhang, and James Z Wang. 2017. Microexpression identification and categorization using a facial dynamics map. IEEE Transactions on Affective Computing. 8(2): p. 254-267.Google ScholarDigital Library
- Zhang, Shiyu, Bailan Feng, Zhineng Chen, 2017 Micro-expression recognition by aggregating local spatio-temporal patterns. in International Conference on Multimedia Modeling. Springer.Google ScholarCross Ref
- Kim, Dae Hoe, Wissam J Baddar, and Yong Man Ro.2016 Micro-expression recognition with expression-state constrained spatio-temporal feature representations. in Proceedings of the 24th ACM international conference on Multimedia.Google Scholar
- Gan, Yee Siang, Sze-Teng Liong, Wei-Chuen Yau, 2019. OFF-ApexNet on micro-expression recognition system. Signal Processing: Image Communication. 74: p. 129-139.Google ScholarDigital Library
- Liong, Sze-Teng, Yee Siang Gan, John See, 2019 Shallow triple stream three-dimensional cnn (ststnet) for micro-expression recognition. in 2019 14th IEEE international conference on automatic face & gesture recognition (FG 2019). IEEE.Google ScholarDigital Library
- Wei, Mengting, Wenming Zheng, Yuan Zong, 2022 A novel micro-expression recognition approach using attention-based magnification-adaptive networks. in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE.Google ScholarCross Ref
- Jaderberg, Max, Karen Simonyan, and Andrew Zisserman. 2015. Spatial transformer networks. Advances in neural information processing systems. 28.Google Scholar
- Mnih, Volodymyr, Nicolas Heess, and Alex Graves. 2014. Recurrent models of visual attention. Advances in neural information processing systems. 27.Google Scholar
- Hu, Jie, Li Shen, and Gang Sun.2018 Squeeze-and-excitation networks. in Proceedings of the IEEE conference on computer vision and pattern recognition.Google Scholar
- Woo, Sanghyun, Jongchan Park, Joon-Young Lee, 2018 Cbam: Convolutional block attention module. in Proceedings of the European conference on computer vision (ECCV).Google ScholarDigital Library
- Zach, Christopher, Thomas Pock, and Horst Bischof.2007 A duality based approach for realtime tv-l 1 optical flow. in Pattern Recognition: 29th DAGM Symposium, Heidelberg, Germany, September 12-14, 2007. Proceedings 29. Springer.Google Scholar
- Davison, Adrian K, Cliff Lansley, Nicholas Costen, 2016. Samm: A spontaneous micro-facial movement dataset. IEEE transactions on affective computing. 9(1): p. 116-129.Google Scholar
- Yan, Wen-Jing, Xiaobai Li, Su-Jing Wang, 2014. CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PloS one. 9(1): p. e86041.Google ScholarCross Ref
- Li, Jingting, Zizhao Dong, Shaoyuan Lu, 2022. CAS (ME) 3: A Third Generation Facial Spontaneous Micro-Expression Database with Depth Information and High Ecological Validity. IEEE Transactions on Pattern Analysis and Machine Intelligence.Google Scholar
- See, John, Moi Hoon Yap, Jingting Li, 2019 Megc 2019–the second facial micro-expressions grand challenge. in 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). IEEE.Google Scholar
- Zhao, Guoying and Matti Pietikainen. 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE transactions on pattern analysis and machine intelligence. 29(6): p. 915-928.Google Scholar
- Liong, Sze-Teng, John See, KokSheik Wong, 2018. Less is more: Micro-expression recognition from video using apex frame. Signal Processing: Image Communication. 62: p. 82-92.Google ScholarCross Ref
Index Terms
- A Micro-expression Recognition Algorithm based on Dense Optical Flow and C3D Network
Recommendations
Micro-expression recognition based on SqueezeNet and C3D
AbstractMicro-expression recognition has attracted extensive attention from psychological and computer vision communities due to its multiple real-life applications. Compared with macro-expression, the change of micro-expression is subtle and difficult ...
Micro-expression recognition based on 3D flow convolutional neural network
AbstractMicro-expression recognition (MER) is a growing field of research which is currently in its early stage of development. Unlike conventional macro-expressions, micro-expressions occur at a very short duration and are elicited in a spontaneous ...
Analysis of Eyebrow Motion for Micro-Expression Recognition
ICIGP '22: Proceedings of the 2022 5th International Conference on Image and Graphics ProcessingFacial expression recognition is a field of emotion recognition that has received considerable attention over the last decades. However, most of the research has been focused on the recognition of macro-expressions that are generally posed and easy to ...
Comments