Skip to main content
Springer Nature Link
Log in

Towards Micro-expression Recognition Through Pyramid of Uniform Temporal Local Binary Pattern Features

  • Conference paper
  • First Online:
Intelligent Systems Design and Applications (ISDA 2018 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 940))

Abstract

Compared to macro-expressions, recognizing micro-expres-sions is more challenging due to low intensity and their brief duration. To deal with this issue, the present paper proposes a facial micro-expression recognition approach based on the pyramid of uniform Temporal Local Binary Pattern (PTLBP\(^{u2}\)) features for describing the appearance motion changes in time through video stream. Unlike the majority of approaches that use a high dimensional feature space, the proposed approach is based on a low dimensional space with only 83 features. Compared to the most recent facial micro-expression recognition approaches, our approach proves its effectiveness with an accuracy rate reaching 66.40% on Casme II dataset. A study of the ability of a macro-expression model to recognize micro-expression shows that it is more efficient to recognize certain micro-expressions than others.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Abdallah, T.B., Guermazi, R., Hammami, M.: Facial-expression recognition based on a low-dimensional temporal feature space. Multimedia Tools Appl. 77(15), 19455–19479 (2018)

    Article  Google Scholar 

  2. Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdisc. Rev. Comput. Stat. 2(4), 443–459 (2010)

    Article  Google Scholar 

  3. Beauchemin, S.S., Barron, J.L.: The computation of optical flow. J. ACM Comput. Surv. 27(3), 433–466 (1995)

    Article  Google Scholar 

  4. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996). https://doi.org/10.1023/A:1018054314350

    Article  Google Scholar 

  5. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  6. Cootes, T., Taylor, C., Cooper, D., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  7. Denisko, D., Hoffman, M.M.: Classification and interaction in random forests. Proc. Nat. Acad. Sci. 115(8), 1690–1692 (2018)

    Article  Google Scholar 

  8. Duan, X., Dai, Q., Wang, X., Wang, Y., Hua, Z.: Recognizing spontaneous micro-expression from eye region. Neurocomputing 217, 27–36 (2016). sI: ALLSHC

    Article  Google Scholar 

  9. Ekman, P.: Telling Lies – Clues to Deceit in the Marketplace, Politics and Marriage 3e (2009)

    Google Scholar 

  10. Goshtasby, A.: Image registration by local approximation methods. Image Vis. Comput. 6(4), 255–261 (1988)

    Article  Google Scholar 

  11. Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)

    Article  Google Scholar 

  12. Huang, X., Wang, S.J., Zhao, G., Piteikainen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), ICCVW 2015, pp. 1–9. IEEE Computer Society, Washington, DC (2015)

    Google Scholar 

  13. Huang, X., Wang, S., Liu, X., Zhao, G., Feng, X., Pietikäinen, M.: Spontaneous facial micro-expression recognition using discriminative spatiotemporal local binary pattern with an improved integral projection. CoRR abs/1608.02255 (2016). http://arxiv.org/abs/1608.02255

  14. Huang, X., Zhao, G., Hong, X., Zheng, W., Pietikäinen, M.: Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomput. 175(PA), 564–578 (2016)

    Article  Google Scholar 

  15. IMOTIONS - BIOMETRIC RESEARCH PLATFORM: Facial expression analysis: the complete pocket guide (2016). https://imotions.com/blog/facial-expression-analysis

  16. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)

    Google Scholar 

  17. Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput. 13(3), 637–649 (2001)

    Article  Google Scholar 

  18. Levina, E., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Advances in Neural Information Processing Systems, Cambridge, MA, USA, pp. 777–784 (2004)

    Google Scholar 

  19. Liong, S., See, J., Phan, R.C., Wong, K.: Less is more: micro-expression recognition from video using apex frame. J. Sig. Process. Image Commun. 62, 82–92 (2018)

    Article  Google Scholar 

  20. Liu, Y.J., Zhang, J.K., Yan, W.J., Wang, S.J., Zhao, G., Fu, X.: A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans. Affect. Comput. 7(4), 299–310 (2015)

    Article  Google Scholar 

  21. Lu, H., Kpalma, K., Ronsin, J.: Motion descriptors for micro-expression recognition. Sig. Process. Image Commun. 67, 108–117 (2018)

    Article  Google Scholar 

  22. Lu, Z., Luo, Z., Zheng, H., Chen, J., Li, W.: A delaunay-based temporal coding model for micro-expression recognition. In: Jawahar, C., Shan, S. (eds.) Computer Vision - ACCV 2014 Workshops, pp. 698–711. Springer International Publishing, Cham (2015)

    Chapter  Google Scholar 

  23. Oh, Y.H., See, J., Le Ngo, A.C., Phan, R.C.W., Baskaran, V.M.: A survey of automatic facial micro-expression analysis: databases, methods, and challenges. Front. Psychol. 9, 11–28 (2018)

    Article  Google Scholar 

  24. O’Sullivan, M., Frank, M.G., Hurley, C.M., Tiwana, J.: Police lie detection accuracy: the effect of lie scenario. J. Law Hum. Behav. 33(6) (2009)

    Google Scholar 

  25. Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: Proceedings of the 13th ACM International Conference on Multimedia (2005)

    Google Scholar 

  26. Platt, J.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Vector Learning. MIT Press (1998)

    Google Scholar 

  27. Ruiz-Hernandez, J.A., Pietikäinen, M.: Encoding local binary patterns using the re-parametrization of the second order Gaussian jet. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6 (2013)

    Google Scholar 

  28. Verikas, A., Gelzinis, A., Bacauskiene, M.: Mining data with random forests: a survey and results of new tests. J. Pattern Recogn. 44(2), 330–49 (2011)

    Article  Google Scholar 

  29. Wang, S.J., Yan, W.J., Li, X., Zhao, G., Fu, X.: Micro-expression recognition using dynamic textures on tensor independent color space. In: 2014 22nd International Conference on Pattern Recognition, pp. 4678–4683 (2014)

    Google Scholar 

  30. Wang, S.J., Chen, H.L., Yan, W.J., Chen, Y.H., Fu, X.: Face recognition and micro-expression recognition based on discriminant tensor subspace analysis plus extreme learning machine. Neural Process. Lett. 39(1), 25–43 (2014)

    Article  Google Scholar 

  31. Wang, Y., See, J., Oh, Y.H., Phan, R.C.W., Rahulamathavan, Y., Ling, H.C., Tan, S.W., Li, X.: Effective recognition of facial micro-expressions with video motion magnification. Multimedia Tools Appl. 76(20), 21665–21690 (2017)

    Article  Google Scholar 

  32. Wang, Y., See, J., Phan, R.C.W., Oh, Y.H.: LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition. In: Cremers, D., Reid, I., Saito, H., Yang, M.H. (eds.) Computer Vision - ACCV 2014, pp. 525–537. Springer International Publishing, Cham (2015)

    Chapter  Google Scholar 

  33. Wolf, L.: Face recognition, geometric vs. appearance-based, pp. 347–352. Springer, Boston (2009)

    Google Scholar 

  34. Xu, F., Zhang, J., Wang, J.Z.: Microexpression identification and categorization using a facial dynamics map. IEEE Trans. Affect. Comput. 8(2), 254–267 (2017)

    Article  Google Scholar 

  35. Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., Fu, X.: CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLOS ONE 9(1), 1–8 (2014)

    Article  Google Scholar 

  36. Zhang, P., Ben, X., Yan, R., Wu, C., Guo, C.: Micro-expression recognition system. Optik Int. J. Light Electron Opt. 127(3), 1395–1400 (2016)

    Article  Google Scholar 

  37. Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)

    Article  Google Scholar 

  38. Zheng, H.: Micro-expression recognition based on 2D Gabor filter and sparse representation. J. Phys. Conf. Ser. 787(1) (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Economics and Management, MIR@CL, University of Sfax, Sfax, Tunisia

    Taoufik Ben Abdallah

  2. Saudi Electronic University, Riyadh, Kingdom of Saudi Arabia

    Radhouane Guermazi

  3. Faculty of Sciences, MIR@CL, University of Sfax, Sfax, Tunisia

    Mohamed Hammami

Authors
  1. Taoufik Ben Abdallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radhouane Guermazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Hammami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taoufik Ben Abdallah .

Editor information

Editors and Affiliations

  1. Machine Intelligence Research Labs, Auburn, WA, USA

    Ajith Abraham

  2. School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India

    Aswani Kumar Cherukuri

  3. Tijuana Institute of Technology, Tijuana, Mexico

    Patricia Melin

  4. Machine Intelligence Research Labs, Auburn, WA, USA

    Niketa Gandhi

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ben Abdallah, T., Guermazi, R., Hammami, M. (2020). Towards Micro-expression Recognition Through Pyramid of Uniform Temporal Local Binary Pattern Features. In: Abraham, A., Cherukuri, A.K., Melin, P., Gandhi, N. (eds) Intelligent Systems Design and Applications. ISDA 2018 2018. Advances in Intelligent Systems and Computing, vol 940. Springer, Cham. https://doi.org/10.1007/978-3-030-16657-1_59

Download citation

Publish with us

Policies and ethics