Skip to main content
SpringerLink
Log in

Micro-expression recognition: an updated review of current trends, challenges and solutions

  • Survey
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Micro-expression (ME) recognition has attracted numerous interests within the computer vision circle in different contexts particularly, localization, magnification, and recognition. Challenges in these areas remain relevant due to the nature of ME’s split-second transition with minute intensity levels. In this paper, a comprehensive state-of-the-art analysis of ME recognition and detection challenges are provided. Contemporary solutions are categorized into low-level, mid-level, and high-level solutions with a review of their characteristics and performances. This paper also provides possible extensions to basic methods, highlight, and predict emerging trends. A thorough analysis of mainstream ME datasets is also provided by elucidating each of their advantages and limitations. This survey gives readers an understanding of ME recognition and an appreciation of future research direction in ME recognition systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Zhang, M., Fu, Q., Chen, Y.-H., Fu, X.: Emotional context influences micro-expression recognition. PLoS ONE 9(4), e95018 (2014)

    Article  Google Scholar 

  2. Takalkar, M., Xu, M., Wu, Q., Chaczko, Z.: A survey: facial micro-expression recognition. Multimed. Tools Appl. 77, 301 (2017)

    Google Scholar 

  3. Ekman, P.: Darwin, deception, and facial expression. Ann. N. Y. Acad. Sci. 1000(1), 205–221 (2003)

    Article  Google Scholar 

  4. Ekman, P.: Micro expression training tool (METT) and subtle expression training tool (SETT). Paul Ekman Company, San Francisco (2003)

    Google Scholar 

  5. Li, X., Xiaopeng, H., Moilanen, A., Huang, X., Pfister, T., Zhao, G., Pietikainen, M.: Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans. Affect Comput. PP(99), 1–1 (2017)

    Google Scholar 

  6. Wen-Jing, Y., Wu, Q., Yong-Jin, L., Su-Jing, W., Fu, X.: CASME database: a dataset of spontaneous micro-expressions collected from neutralized faces. In: 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 22–26 April 2013, pp. 1–7

  7. Yan, W.-J., Li, X., Wang, S.-J., Zhao, G., Liu, Y.-J., Chen, Y.-H., Fu, X.: CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1), e86041 (2014)

    Article  Google Scholar 

  8. Li, X., Pfister, T., Huang, X., Zhao, G., Pietikäinen, M.: A spontaneous micro-expression database: inducement, collection and baseline. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 22–26 April 2013, pp. 1–6

  9. Ekman, P., Friesen, W.V.: Facial Action Coding System, vol. 1. Consulting Psychologists Press, Washington (1978)

    Google Scholar 

  10. Tian, Y.I., Kanade, T., Cohn, J.F.: Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)

    Article  Google Scholar 

  11. Frank, M., Herbasz, M., Sinuk, K., Keller, A., Nolan, C.: I see how you feel: training laypeople and professionals to recognize fleeting emotions. In: The Annual Meeting of the International Communication Association. Sheraton New York, New York City (2009)

  12. Yan, W.-J., Wu, Q., Liang, J., Chen, Y.-H., Fu, X.: How fast are the leaked facial expressions: the duration of micro-expressions. J. Nonverbal Behav. 37(4), 217–230 (2013)

    Article  Google Scholar 

  13. Xu, F., Zhang, J., Wang, J.Z.: Microexpression identification and categorization using a facial dynamics map. IEEE Trans. Affect Comput. 8(2), 254–267 (2017)

    Article  Google Scholar 

  14. Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996)

    Article  Google Scholar 

  15. Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)

    Article  Google Scholar 

  16. Shreve, M., Brizzi, J., Fefilatyev, S., Luguev, T., Goldgof, D., Sarkar, S.: Automatic expression spotting in videos. Image Vis. Comput. 32(8), 476–486 (2014)

    Article  Google Scholar 

  17. Qu, F., Wang, S.J., Yan, W.J., Li, H., Wu, S., Fu, X.: CAS(ME)2: a database for spontaneous macro-expression and micro-expression spotting and recognition. IEEE Trans. Affect Comput. PP(99), 1–1 (2017). https://doi.org/10.1109/taffc.2017.2654440

    Article  Google Scholar 

  18. Yan, W.-J., Wang, S.-J., Chen, Y.-H., Zhao, G., Fu, X.: Quantifying micro-expressions with constraint local model and local binary pattern. In: Cham 2015. Computer Vision—ECCV 2014 Workshops, pp. 296–305. Springer

  19. Porter, S., Brinke, L.T.: Reading between the lies: identifying concealed and falsified emotions in universal facial expressions. Psychol. Sci. 19(5), 508–514 (2008). https://doi.org/10.1111/j.1467-9280.2008.02116.x

    Article  Google Scholar 

  20. Yan, W.-J., Wang, S.-J., Liu, Y.-J., Wu, Q., Fu, X.: For micro-expression recognition: database and suggestions. Neurocomputing 136, 82–87 (2014)

    Article  Google Scholar 

  21. Wang, Y., See, J., Oh, Y.-H., Phan, R.C.-W., Rahulamathavan, Y., Ling, H.-C., Tan, S.-W., Li, X.: Effective recognition of facial micro-expressions with video motion magnification. Multimed. Tools Appl. 76(20), 21665–21690 (2017)

    Article  Google Scholar 

  22. Ngo, A.C.L., Oh, Y.H., Phan, R.C.W., See, J.: Eulerian emotion magnification for subtle expression recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 20–25 March 2016, pp. 1243–1247

  23. Le Ngo, A.C., Phan, R.C.-W., See, J.: Spontaneous subtle expression recognition: imbalanced databases and solutions. In: Cham 2015. Computer Vision—ACCV 2014, pp. 33–48. Springer

  24. Polikovsky, S., Kameda, Y., Ohta, Y.: Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. In: 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP 2009), 3–3 Dec 2009, pp. 1–6

  25. Shreve, M., Godavarthy, S., Goldgof, D., Sarkar, S.: Macro- and micro-expression spotting in long videos using spatio-temporal strain. Face Gesture 2011(21–25), 51–56 (2011)

    Google Scholar 

  26. Warren, G., Schertler, E., Bull, P.: Detecting deception from emotional and unemotional cues. J. Nonverbal Behav. 33(1), 59–69 (2009)

    Article  Google Scholar 

  27. Davison, A.K., Lansley, C., Costen, N., Tan, K., Yap, M.H.: SAMM: a spontaneous micro-facial movement dataset. IEEE Trans. Affect Comput. PP(99), 1–1 (2016)

    Google Scholar 

  28. Yu, Z., Liu, Q., Liu, G.: Deeper cascaded peak-piloted network for weak expression recognition. Vis. Comput. 1–9 (2017)

  29. Polikovsky, S., Kameda, Y., Ohta, Y.: Facial micro-expression detection in hi-speed video based on facial action coding system (FACS). IEICE Trans. Inf. Syst. E96.D(1), 81–92 (2013)

    Article  Google Scholar 

  30. Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)

    Article  Google Scholar 

  31. Sariyanidi, E., Gunes, H., Cavallaro, A.: Robust registration of dynamic facial sequences. IEEE Trans. Image Process. 26(4), 1708–1722 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Wang, Y., See, J., Phan, R.C.W., Oh, Y.-H.: Efficient spatio-temporal local binary patterns for spontaneous facial micro-expression recognition. PLoS ONE 10(5), e0124674 (2015)

    Article  Google Scholar 

  33. Le Ngo, A.C., See, J., Phan, R.C.W.: Sparsity in dynamics of spontaneous subtle emotions: analysis and application. IEEE Trans. Affect Comput. 8(3), 396–411 (2017)

    Article  Google Scholar 

  34. Liong, S.-T., See, J., Phan, R.C.-W., Le Ngo, A.C., Oh, Y.-H., Wong, K.: Subtle expression recognition using optical strain weighted features. In: Cham 2015. Computer Vision—ACCV 2014 Workshops, pp. 644–657. Springer

  35. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 2001, vol. 511, pp. I-511–I-518

  36. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, 16–21 June 2012, pp. 2879–2886

  37. Chen, D., Ren, S., Wei, Y., Cao, X., Sun, J.: Joint cascade face detection and alignment. In: Cham 2014. Computer Vision—ECCV 2014, pp. 109–122. Springer

  38. Matsugu, M., Mori, K., Mitari, Y., Kaneda, Y.: Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Netw. 16(5), 555–559 (2003)

    Article  Google Scholar 

  39. Liao, S., Jain, A.K., Li, S.Z.: A fast and accurate unconstrained face detector. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 211–223 (2016)

    Article  Google Scholar 

  40. Ranjan, R., Patel, V.M., Chellappa, R.: A deep pyramid deformable part model for face detection. In: IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), 8–11 Sept 2015, pp. 1–8

  41. Yang, S., Luo, P., Loy, C.C., Tang, X.: From facial parts responses to face detection: a deep learning approach. In: IEEE International Conference on Computer Vision (ICCV), 7–13 Dec 2015, pp. 3676–3684

  42. Li, H., Lin, Z., Shen, X., Brandt, J., Hua, G.: A convolutional neural network cascade for face detection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7–12 June 2015, pp. 5325–5334 (2015)

  43. Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1–1 (2017)

    Google Scholar 

  44. Yan, J., Zhang, X., Lei, Z., Li, S.Z.: Face detection by structural models. Image Vis. Comput. 32(10), 790–799 (2014)

    Article  Google Scholar 

  45. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  46. Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Recognising spontaneous facial micro-expressions. In: IEEE International Conference on Computer Vision (ICCV) 2011, pp. 1449–1456. IEEE

  47. Wang, S.-J., Yan, W.-J., Zhao, G., Fu, X., Zhou, C.-G.: Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) Computer Vision—ECCV 2014 Workshops: Zurich, Switzerland, September 6–7 and 12, 2014, Proceedings, Part I. pp. 325–338. Springer, Cham (2015)

  48. Ruiz-Hernandez, J.A., Pietikäinen, M.: Encoding local binary patterns using the re-parametrization of the second order Gaussian jet. In: 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 22–26 April 2013, pp. 1–6

  49. Wang, S.J., Yan, W.J., Li, X., Zhao, G., Zhou, C.G., Fu, X., Yang, M., Tao, J.: Micro-expression recognition using color spaces. IEEE Trans. Image Process. 24(12), 6034–6047 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  50. Wang, S.J., Yan, W.J., Li, X., Zhao, G., Fu, X.: Micro-expression recognition using dynamic textures on tensor independent color space. In: 22nd International Conference on Pattern Recognition, 24–28 Aug 2014, pp. 4678–4683

  51. Huang, X., Zhao, G., Hong, X., Zheng, W., Pietikäinen, M.: Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing 175, 564–578 (2016)

    Article  Google Scholar 

  52. Huang, X.H., Wang, S.J., Liu, X., Zhao, G., Feng, X., Pietikainen, M.: Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition. IEEE Trans. Affect Comput. PP(99), 1–1 (2017)

    Google Scholar 

  53. Xia, Z., Feng, X., Peng, J., Peng, X., Zhao, G.: Spontaneous micro-expression spotting via geometric deformation modeling. Comput. Vis. Image Underst. 147, 87–94 (2016)

    Article  Google Scholar 

  54. Huang, X., Wang, S.J., Zhao, G., Piteikäinen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: IEEE International Conference on Computer Vision Workshop (ICCVW), 7–13 Dec 2015, pp. 1–9

  55. Liu, Q., Deng, J., Tao, D.: Dual sparse constrained cascade regression for robust face alignment. IEEE Trans. Image Process. 25(2), 700–712 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  56. Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7–12 June 2015, pp. 3659–3667

  57. Xiong, X., Torre, F.D.L.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, 23–28 June 2013, pp. 532–539

  58. Gines Hidalgo, Z.C., Tomas, S., Shih-En, W., Hanbyul, J., Yaser, S.: OpenPose Library. https://github.com/CMU-Perceptual-Computing-Lab/openpose (2017). 2018

  59. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: CVPR 2017, p. 4

  60. Deng, W., Fang, Y., Xu, Z., Hu, J.: Facial landmark localization by enhanced convolutional neural network. Neurocomputing 273, 222–229 (2018)

    Article  Google Scholar 

  61. Bian, P., Xie, Z., Jin, Y.: Multi-task feature learning-based improved supervised descent method for facial landmark detection. SIViP 12(1), 17–24 (2018)

    Article  Google Scholar 

  62. Pan, W., Qin, K., Chen, Y.: An adaptable-multilayer fractional fourier transform approach for image registration. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 400–414 (2009)

    Article  Google Scholar 

  63. Kumar, S., Azartash, H., Biswas, M., Nguyen, T.: Real-time affine global motion estimation using phase correlation and its application for digital image stabilization. IEEE Trans. Image Process. 20(12), 3406–3418 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  64. Xiong, X., Qin, K.: Linearly estimating all parameters of affine motion using radon transform. IEEE Trans. Image Process. 23(10), 4311–4321 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  65. Lucas, B.D.: Generalized image matching by the method of differences. (1985)

  66. Baker, S., Matthews, I.: Lucas–Kanade 20 years on: a unifying framework. Int. J. Comput. Vis. 56(3), 221–255 (2004)

    Article  Google Scholar 

  67. Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: Robust and efficient parametric face alignment. In: International Conference on Computer Vision, 6–13 Nov 2011, pp. 1847–1854

  68. Evangelidis, G.D., Psarakis, E.Z.: Parametric image alignment using enhanced correlation coefficient maximization. IEEE Trans. Pattern Anal. Mach. Intell. 30(10), 1858–1865 (2008)

    Article  Google Scholar 

  69. Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 6–13 Nov 2011, pp. 868–875

  70. Gunn, S.R.: Support vector machines for classfication and regression. ISIS Tech. Rep. 14(1), 5–16 (1998)

    Google Scholar 

  71. Guo, Y., Tian, Y., Gao, X., Zhang, X.: Micro-expression recognition based on local binary patterns from three orthogonal planes and nearest neighbor method. In: 2014 International Joint Conference on Neural Networks (IJCNN), 6–11 July 2014, pp. 3473–3479

  72. Happy, S.L., Routray, A.: Fuzzy histogram of optical flow orientations for micro-expression recognition. IEEE Trans. Affect Comput. PP(99), 1–1 (2017)

    Google Scholar 

  73. Ben, X., Zhang, P., Yan, R., Yang, M., Ge, G.: Gait recognition and micro-expression recognition based on maximum margin projection with tensor representation. Neural Comput. Appl. 27(8), 2629–2646 (2016)

    Article  Google Scholar 

  74. Wang, S.-J., Chen, H.-L., Yan, W.-J., Chen, Y.-H., Fu, X.: Face recognition and micro-expression recognition based on discriminant tensor subspace analysis plus extreme learning machine. Neural Process. Lett. 39(1), 25–43 (2014)

    Article  Google Scholar 

  75. Adegun, I.P., Vadapalli, H.B.: Automatic recognition of micro-expressions using local binary patterns on three orthogonal planes and extreme learning machine. In: Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech), 30 Nov 2016–2 Dec 2016, pp. 1–5

  76. Guo, Y., Xue, C., Wang, Y., Yu, M.: Micro-expression recognition based on CBP-TOP feature with ELM. Opt. Int. J. Light Electron Opt. 126(23), 4446–4451 (2015)

    Article  Google Scholar 

  77. Izenman, A.J.: Linear discriminant analysis. In: Casella, G., Fienberg, S., Olkin, I. (eds.) Modern Multivariate Statistical Techniques, pp. 237–280. Springer, Cham (2013)

    Chapter  Google Scholar 

  78. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)

    MATH  Google Scholar 

  79. Patel, D., Hong, X., Zhao, G.: Selective deep features for micro-expression recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), 4–8 Dec 2016, pp. 2258–2263

  80. Kim, D.H., Baddar, W., Jang, J., Ro, Y.M.: Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition. IEEE Trans. Affect Comput. 66(99), 1–1 (2017)

    Google Scholar 

  81. Takalkar, M.A., Xu, M.: Image based facial micro-expression recognition using deep learning on small datasets. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA), 29 Nov 2017–1 Dec 2017, pp. 1–7

  82. Lim, C.H., Goh, K.M.: Fuzzy qualitative approach for micro-expression recognition. In: Asic-Pasific Signal and Information Processing Association Annual Submit and Conference Kuala Lumpur, 12nd–15th Dec 2017

  83. Lim, C.H., Risnumawan, A., Chan, C.S.: A scene image is nonmutually exclusive—a fuzzy qualitative scene understanding. IEEE Trans. Fuzzy Syst. 22(6), 1541–1556 (2014)

    Article  Google Scholar 

  84. Liu, H., Coghill, G.M., Barnes, D.P.: Fuzzy qualitative trigonometry. Int. J. Approx. Reason. 51(1), 71–88 (2009)

    Article  MATH  Google Scholar 

  85. Shen, Q., Leitch, R.: Fuzzy qualitative simulation. IEEE Trans. Syst. Man Cybern. 23(4), 1038–1061 (1993)

    Article  Google Scholar 

  86. Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)

    Article  Google Scholar 

  87. House, C., Meyer, R.: Preprocessing and descriptor features for facial micro-expression recognition (2015)

  88. Wang, Y., See, J., Phan, R.C.-W., Oh, Y.-H.: LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition. In: Cham 2015. Computer Vision—ACCV 2014, pp. 525–537. Springer

  89. Mayya, V., Pai, R.M., Pai, M.M.M.: Combining temporal interpolation and DCNN for faster recognition of micro-expressions in video sequences. In: International Conference on Advances in Computing, Communications and Informatics (ICACCI), 21–24 Sept 2016, pp. 699–703

  90. Lee, S.H., Kim, H., Ro, Y.M., Plataniotis, K.N.: Using color texture sparsity for facial expression recognition. In: 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 22–26 April 2013, pp. 1–6

  91. Liong, S.-T., Phan, R.C.-W., See, J., Oh, Y.-H., Wong, K.: Optical strain based recognition of subtle emotions. In: International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS) 2014, pp. 180–184. IEEE

  92. Liong, S.-T., See, J., Phan, R.C.W., Oh, Y.-H., Cat Le Ngo, A., Wong, K., Tan, S.-W.: Spontaneous subtle expression detection and recognition based on facial strain. Signal Process. Image Commun. 47, 170–182 (2016)

    Article  Google Scholar 

  93. Ben, X., Jia, X., Yan, R., Zhang, X., Meng, W.: Learning effective binary descriptors for micro-expression recognition transferred by macro-information. Pattern Recogn. Lett. 107, 50–58 (2017)

    Article  Google Scholar 

  94. Chaudhry, R., Ravichandran, A., Hager, G., Vidal, R.: Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: IEEE Conference on Computer Vision and Pattern Recognition, 20–25 June 2009, pp. 1932–1939

  95. Liong, S.-T., See, J., Wong, K., Phan, R.C.-W.: Less is more: micro-expression recognition from video using apex frame. Signal Process. Image Commun. 62, 82–92 (2018)

    Article  Google Scholar 

  96. Liu, Y.J., Zhang, J.K., Yan, W.J., Wang, S.J., Zhao, G., Fu, X.: A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans. Affect Comput. 7(4), 299–310 (2016)

    Article  Google Scholar 

  97. Oh, Y.H., Ngo, A.C.L., See, J., Liong, S.T., Phan, R.C.W., Ling, H.C.: Monogenic Riesz wavelet representation for micro-expression recognition. In: IEEE International Conference on Digital Signal Processing (DSP), 21–24 July 2015, pp. 1237–1241

  98. Zheng, H.: Micro-expression recognition based on 2D Gabor filter and sparse representation. J. Phys. Conf. Ser. 787(1), 012013 (2017)

    Article  Google Scholar 

  99. Chengjun, L., Wechsler, H.: Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans. Image Process. 11(4), 467–476 (2002)

    Article  Google Scholar 

  100. Kamarol, S.K.A., Jaward, M.H., Parkkinen, J., Parthiban, R.: Spatiotemporal feature extraction for facial expression recognition. IET Image Process. 10, 534 (2016)

    Article  Google Scholar 

  101. Lu, Z., Luo, Z., Zheng, H., Chen, J., Li, W.: A Delaunay-based temporal coding model for micro-expression recognition. In: Cham 2015. Computer Vision—ACCV 2014 Workshops, pp. 698–711. Springer

  102. Sariyanidi, E., Gunes, H., Cavallaro, A.: Learning bases of activity for facial expression recognition. IEEE Trans. Image Process. 26(4), 1965–1978 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  103. He, J., Hu, J.-F., Lu, X., Zheng, W.-S.: Multi-task mid-level feature learning for micro-expression recognition. Pattern Recognit. 66, 44–52 (2017)

    Article  Google Scholar 

  104. Sikka, K., Wu, T., Susskind, J., Bartlett, M.: Exploring bag of words architectures in the facial expression domain. In: Berlin, Heidelberg 2012. Computer Vision—ECCV 2012. Workshops and Demonstrations, pp. 250–259. Springer, Berlin

  105. Zheng, H., Geng, X., Yang, Z.: A Relaxed K-SVD algorithm for spontaneous micro-expression recognition. In: Cham 2016. PRICAI 2016: Trends in Artificial Intelligence, pp. 692–699. Springer

  106. Zheng, H., Zhu, J., Yang, Z., Jin, Z.: Effective micro-expression recognition using relaxed K-SVD algorithm. Int. J. Mach. Learn. Cybern. 8(6), 2043–2049 (2017)

    Article  Google Scholar 

  107. Pei, L., Ye, M., Zhao, X., Dou, Y., Bao, J.: Action recognition by learning temporal slowness invariant features. Vis. Comput. 32(11), 1395–1404 (2016)

    Article  Google Scholar 

  108. Wu, H., Zou, B., Zhao, Y.-Q., Guo, J.: Scene text detection using adaptive color reduction, adjacent character model and hybrid verification strategy. Vis. Comput. 33(1), 113–126 (2017)

    Article  Google Scholar 

  109. Bi, L., Kim, J., Kumar, A., Fulham, M., Feng, D.: Stacked fully convolutional networks with multi-channel learning: application to medical image segmentation. Vis. Comput. 33(6), 1061–1071 (2017)

    Article  Google Scholar 

  110. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the Devil in the Details: Delving Deep into Convolutional Nets. CoRR abs/1405.3531 (2014)

  111. Peng, M., Wang, C., Chen, T., Liu, G., Fu, X.: Dual temporal scale convolutional neural network for micro-expression recognition. Front. Psychol. 8, 1745 (2017)

    Article  Google Scholar 

  112. Hong, X., Tran, T.-K., Zhao, G.: Micro-Expression Spotting: A Benchmark. CoRR abs/1710.02820 (2017)

  113. Tran, T.-K., Hong, X., Zhao, G.: Sliding window based micro-expression spotting: a benchmark. In: Cham 2017. Advanced Concepts for Intelligent Vision Systems, pp. 542–553. Springer

  114. Wu, Q., Shen, X., Fu, X.: The machine knows what you are hiding: an automatic micro-expression recognition system. In: Berlin, Heidelberg 2011. Affective Computing and Intelligent Interaction, pp. 152–162. Springer, Berlin

  115. Moilanen, A., Zhao, G., Pietikäinen, M.: Spotting rapid facial movements from videos using appearance-based feature difference analysis. In: 22nd International Conference on Pattern Recognition, 24–28 Aug 2014, pp. 1722–1727

  116. Davison, A.K., Yap, M.H., Lansley, C.: Micro-facial movement detection using individualised baselines and histogram-based descriptors. In: IEEE International Conference on Systems, Man, and Cybernetics, 9–12 Oct 2015, pp. 1864–1869

  117. Davison, A.K., Lansley, C., Ng, C.-C., Tan, K., Yap, M.H.: Objective micro-facial movement detection using FACS-based regions and baseline evaluation. CoRR abs/1612.05038 (2016)

  118. Lu, H., Kpalma, K., Ronsin, J.: Micro-expression detection using integral projections. J WSCG 25(2), 87–96 (2017)

    Google Scholar 

  119. Shreve, M., Godavarthy, S., Manohar, V., Goldgof, D., Sarkar, S.: Towards macro- and micro-expression spotting in video using strain patterns. In: Workshop on Applications of Computer Vision (WACV), 7–8 Dec 2009, pp. 1–6

  120. Wang, S.-J., Wu, S., Qian, X., Li, J., Fu, X.: A main directional maximal difference analysis for spotting facial movements from long-term videos. Neurocomputing 230, 382–389 (2017)

    Article  Google Scholar 

  121. Li, X., Yu, J., Zhan, S.: Spontaneous facial micro-expression detection based on deep learning. In: IEEE 13th International Conference on Signal Processing (ICSP), 6–10 Nov 2016, pp. 1130–1134

  122. Duque, C., Alata, O., Emonet, R., Legrand, A.-C., Konik, H.: Micro-expression spotting using the Riesz pyramid. In: WACV 2018 (2018)

  123. Wang, S.-J., Yan, W.-J., Zhao, G., Fu, X., Zhou, C.-G.: Micro-expression recognition using robust principal component analysis and local spatiotemporal directional features. In: Cham 2015. Computer Vision—ECCV 2014 Workshops, pp. 325–338. Springer

  124. Parkinson, C., Walker, T.T., Memmi, S., Wheatley, T.: Emotions are understood from biological motion across remote cultures. Emotion 17(3), 459–477 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Ministry of Higher Education (MOHE), Malaysia, under Fundamental Research Grant Scheme (FRGS) (Reference code: FRGS/1/2016/TK04/TARUC/02/1).

Author information

Authors and Affiliations

  1. Department of Electrical and Electronics Engineering, Faculty of Engineering and Technology, Tunku Abdul Rahman University College, Jalan Genting Kelang, 53300, Wilayah Persekutuan Kuala Lumpur, Malaysia

    Kam Meng Goh, Chee How Ng & Li Li Lim

  2. Faculty of Electrical Engineering, Universiti Teknologi Malaysia, 81310, Skudai, Johor, Malaysia

    U. U. Sheikh

Authors
  1. Kam Meng Goh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chee How Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Li Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. U. U. Sheikh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kam Meng Goh.

Ethics declarations

Conflict of interest

The author declares that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goh, K.M., Ng, C.H., Lim, L.L. et al. Micro-expression recognition: an updated review of current trends, challenges and solutions. Vis Comput 36, 445–468 (2020). https://doi.org/10.1007/s00371-018-1607-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1607-6

Keywords

Search

Navigation