skip to main content
10.1145/3474085.3479225acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

Research on Micro-Expression Spotting Method Based on Optical Flow Features

Published: 17 October 2021 Publication History

Abstract

This paper aims to propose an automatic micro-expression spotting method of high accuracy and high robustness. Due to the characteristics of small amplitude and short duration, how to accurately capture the subtle movements of micro-expression is a complex problem. The optical flow method is applied to estimate the motion trend of the facial regions. Because the head shaking is an essential reason for the high false-positive rate of micro-expression spotting, a reliable face alignment method becomes crucial. According to the optical flow of the nose tip region, the cutting box was adjusted several times to optimize the relative position between the face and the cutting box stable. On this basis, the optical flow features from the 14 regions of interest on the face are used to build a feature matrix, and a wave peak location technology is proposed to accurately locate the moment when the micro-expression occurs on the time-domain curve of the features. The experimental results on the CAS(ME)2-cropped and the SAMM Long Videos datasets show that our method performs significantly better than the baseline method and has a high application value in various application scenarios.

Supplementary Material

MP4 File (MM21-gch3314.mp4)
Due to small amplitude and short duration, how to accurately capture the subtle movements of micro-expression is a complex problem. The optical flow method is applied to estimate the motion trend of the facial regions. Because the head shaking is an essential reason for the high false-positive rate of micro-expression spotting, a reliable face alignment method becomes crucial. According to the optical flow of the nose tip region, the cutting box was adjusted several times to optimize the relative position between the face and the cutting box stable. On this basis, the optical flow features from the 14 regions of interest on the face are used to build a feature matrix, and a wave peak location technology is proposed to accurately locate the moment when the micro-expression occurs on the time-domain curve of the features. The experimental results on the CAS(ME)2-cropped and the SAMM Long Videos datasets show that our method performs significantly better than the baseline method.

References

[1]
P. Ekman and W. V. Friesen, ?Nonverbal leakage and clues to deception," Psychiatry, vol. 32, no. 1, pp. 88--106, 1969.
[2]
Liong S T, See J, Wong K S, et al. Automatic Apex Frame Spotting in Micro-expression Database[C], IAPR Asian Conference on Pattern Recognition. IEEE, 2015: 665--669.P. Ekman and W. V. Friesen, "Nonverbal leakage and clues to deception," Psychiatry, vol. 32, no. 1, pp. 88--106, 1969.
[3]
He Y, Wang S J, Li J, et al. Spotting Macro- and Micro-expression Intervals in Long Video Sequences[J]. 2019: 742--748.
[4]
Wang S J, Wu S, Qian X, et al. A main directional maximal difference analysis for spotting facial movements from long-term videos[J]. Neurocomputing, 2017, 230:382--389.
[5]
Zhang Z, Chen T, Meng H, et al. SMEConvNet: A Convolutional Neural Network for Spotting Spontaneous Facial Micro-Expression from Long Videos[J]. IEEE Access, 2018:1--1.
[6]
Zhang L W, Li J, Wang S J, et al. Spatio-temporal fusion for Macro-and Micro-expression Spotting in Long Video Sequences[C], 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (FG). IEEE, 2020:734--741
[7]
D. E. King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10(Jul):1755--1758, 2009.
[8]
Ekman P., Friesen W.V. Facial Action Coding System (FACS): a Technique for the Measurement of Facial Actions[J]. Rivista Di Psichiatria, 1978, 47(2):126--38.
[9]
F. Qu, S.-J. Wang, W.-J. Yan, H. Li, S. Wu, and X. Fu. CAS(ME)2: A database for spontaneous macro-expression and micro-expression spotting and recognition. IEEE Transactions on Affective Computing, 9(4):424--436, 2017.
[10]
C. H. Yap, C. Kendrick, and M. H. Yap. Samm long videos: A spontaneous facial micro-and macro-expressions dataset. arXiv preprint arXiv:1911.01519, 2019.
[11]
Yap, CH, Yap, MH, Davison, AK, Cunningham, R. (2021), Efficient Lightweight 3D-CNN using Frame Skipping and Contrast Enhancement for Facial Macro- and Micro-expression Spotting, arXiv:2105.06340[cs.CV], https: //arxiv.org/abs/2105.06340

Cited By

View all
  • (2024)Enhancing Micro-Expression Analysis Performance by Effectively Addressing Data ImbalanceProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689144(11503-11507)Online publication date: 28-Oct-2024
  • (2024)A Multi-scale Feature Learning Network with Optical Flow Correction for Micro- and Macro-expression SpottingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689143(11497-11502)Online publication date: 28-Oct-2024
  • (2024)Micro-Expression Spotting Based on Optical Flow Feature with Boundary CalibrationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689142(11490-11496)Online publication date: 28-Oct-2024
  • Show More Cited By

Index Terms

  1. Research on Micro-Expression Spotting Method Based on Optical Flow Features

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '21: Proceedings of the 29th ACM International Conference on Multimedia
    October 2021
    5796 pages
    ISBN:9781450386517
    DOI:10.1145/3474085
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 17 October 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. face alignment method
    2. optical flow method
    3. peak location technology

    Qualifiers

    • Short-paper

    Conference

    MM '21
    Sponsor:
    MM '21: ACM Multimedia Conference
    October 20 - 24, 2021
    Virtual Event, China

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)97
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Enhancing Micro-Expression Analysis Performance by Effectively Addressing Data ImbalanceProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689144(11503-11507)Online publication date: 28-Oct-2024
    • (2024)A Multi-scale Feature Learning Network with Optical Flow Correction for Micro- and Macro-expression SpottingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689143(11497-11502)Online publication date: 28-Oct-2024
    • (2024)Micro-Expression Spotting Based on Optical Flow Feature with Boundary CalibrationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3689142(11490-11496)Online publication date: 28-Oct-2024
    • (2024)Facial Prior Guided Micro-Expression GenerationIEEE Transactions on Image Processing10.1109/TIP.2023.334517733(525-540)Online publication date: 1-Jan-2024
    • (2024)Two-Stage Facial Expression Spotting with Spectrum-Based Post-Processing2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10687531(1-6)Online publication date: 15-Jul-2024
    • (2024)SFAMNet: A scene flow attention-based micro-expression networkNeurocomputing10.1016/j.neucom.2023.126998566(126998)Online publication date: Jan-2024
    • (2024)Local and Global Features Interactive Fusion Network for Macro- and Micro-expression Spotting in Long VideosPattern Recognition and Computer Vision10.1007/978-981-97-8795-1_23(336-350)Online publication date: 3-Nov-2024
    • (2024)Learning Interval-Aware Embedding for Macro and Micro-expression SpottingComputer Vision – ACCV 202410.1007/978-981-96-0911-6_22(373-390)Online publication date: 8-Dec-2024
    • (2023)Efficient Micro-Expression Spotting Based on Main Directional Mean Optical Flow FeatureProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612861(9541-9545)Online publication date: 26-Oct-2023
    • (2023)Micro-Expression Spotting with Face Alignment and Optical FlowProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612853(9501-9505)Online publication date: 26-Oct-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media