Skip to main content

Advertisement

SpringerLink
Log in

The design of evolutionary feature selection operator for the micro-expression recognition

  • Regular research paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

The evolutionary algorithm is widely deployed in the feature selection task, but the complexity of the solution for the feature selection problem grows exponentially with the increase of feature dimensions. To handle this problem, this study proposes a novel feature selection operator for the Micro-Expression (ME) recognition task based on Genetic Programming, which can reduce the complexity caused by the high dimensional features. The One versus Rest (OVR) scheme is used to decompose the three-class problem of ME into three binary class problem, so as to reduce the difficulty of the recognition task. A feature selection operator is designed based on the characteristics of ME data, aiming to select class-relevant features from a set of redundant facial features. In our algorithm, the optical flow features are extracted for different ME apex frames firstly, and then these features are split into a set of segments by the sliding windows, which are set as the terminal set. The feature selection operator merges a set of segments to form a new feature subset, and it guides the individuals to evolve towards higher discriminative ability. We test the proposed method on three public data sets (CASME, SAMM, SMIC), and the experimental results show that our method can effectively solve the class imbalance problem and lead to better performance compared with other state-of-the-art algorithms. Further analysis reveals that the feature subsets selected by our algorithms for the three binary class problems contain only 0.57% duplicated features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Yan W-J et al (2013) How fast are the leaked facial expressions: the duration of micro-expressions. J Nonverbal Behav 37(4):217–230

    Article  Google Scholar 

  2. Ekman P, Friesen WV (1971) Constants across cultures in the face and emotion. J Pers Soc Psychol 17(2):124

    Article  Google Scholar 

  3. Liu KH et al (2009) Ensemble component selection for improving ICA based microarray data prediction models. Pattern Recogn 42(7):1274–1283

    Article  MATH  Google Scholar 

  4. Patel D, Hong X, Zhao G (2016) Selective deep features for micro-expression recognition. In: 2016 23rd international conference on pattern recognition (ICPR)

  5. Zhang YP et al (2020) A novel multi-objective genetic algorithm based error correcting output codes. Swarm Evol Comput 57:100709

    Article  Google Scholar 

  6. Hammami M et al (2019) A multi-objective hybrid filter-wrapper evolutionary approach for feature selection. Memetic Comput 11(2):193–208

    Article  Google Scholar 

  7. Ghosh M et al (2019) Feature selection for handwritten word recognition using memetic algorithm. Advances in intelligent computing. Springer, pp 103–124

    Chapter  Google Scholar 

  8. Nekkaa M, Boughaci D (2015) A memetic algorithm with support vector machine for feature selection and classification. Memetic Comput 7(1):59–73

    Article  Google Scholar 

  9. Eskridge BE, Hougen DF (2004) Imitating success: a memetic crossover operator for genetic programming. In: Proceedings of the 2004 congress on evolutionary computation (IEEE Cat. No. 04TH8753)

  10. Cheng T, Zhong J (2020) An efficient memetic genetic programming framework for symbolic regression. Memetic Comput 12(4):299–315

    Article  Google Scholar 

  11. Muñoz L et al (2019) Evolving multidimensional transformations for symbolic regression with M3GP. Memetic Comput 11(2):111–126

    Article  Google Scholar 

  12. Liang J, Xue Y (2021) Multi-objective memetic algorithms with tree-based genetic programming and local search for symbolic regression. Neural Process Lett 53(3):2197–2219

    Article  Google Scholar 

  13. Zhang M, et al (2017) Memetic algorithm based feature selection for hyperspectral images classification. In: 2017 IEEE congress on evolutionary computation (CEC)

  14. Ghosh M et al (2019) Feature selection for facial emotion recognition using late hill-climbing based memetic algorithm. Multimed Tools Appl 78(18):25753–25779

    Article  Google Scholar 

  15. Abu Zaher A et al (2019) An adaptive memetic algorithm for feature selection using proximity graphs. Comput Intell 35(1):156–183

    Article  MathSciNet  Google Scholar 

  16. Liong S-T, et al (2014) Optical strain based recognition of subtle emotions. In: 2014 IEEE international symposium on intelligent signal processing and communication systems (ISPACS)

  17. Liu Y-J et al (2015) A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans Affect Comput 7(4):299–310

    Article  Google Scholar 

  18. Huang X et al (2016) Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing 175:564–578

    Article  Google Scholar 

  19. Li X et al (2017) Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods. IEEE Trans Affect Comput 9(4):563–577

    Article  Google Scholar 

  20. Liong ST, et al (2020) Evaluation of the spatio-temporal features and gan for micro-expression recognition system. J Signal Process Syst 1–21

  21. Liong ST, et al (2019) Shallow triple stream three-dimensional cnn (ststnet) for micro-expression recognition. In: 2019 14th IEEE international conference on automatic face and gesture recognition (FG 2019)

  22. Liong S-T et al (2018) Less is more: micro-expression recognition from video using apex frame. Signal Process Image Commun 62:82–92

    Article  Google Scholar 

  23. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  24. Yan W-J et al (2014) CASME II: an improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE 9(1):e86041

    Article  Google Scholar 

  25. Guo C et al (2019) Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition. IEEE Access 7:174517–174530

    Article  Google Scholar 

  26. Shreve M, et al (2011) Macro-and micro-expression spotting in long videos using spatio-temporal strain. Face and Gesture 2011

  27. Liong S-T, et al (2014) Subtle expression recognition using optical strain weighted features. In: Asian conference on computer vision

  28. Happy S, Routray A (2017) Fuzzy histogram of optical flow orientations for micro-expression recognition. IEEE Trans Affect Comput 10(3):394–406

    Article  Google Scholar 

  29. Peng M, et al (2018) From macro to micro expression recognition: deep learning on small datasets using transfer learning. In: 2018 13th IEEE international conference on automatic face and gesture recognition (FG 2018)

  30. van Quang N, Chun J, Tokuyama T (2019) CapsuleNet for micro-expression recognition. In: 2019 14th IEEE international conference on automatic face and gesture recognition (FG 2019)

  31. Espejo PG, Ventura S, Herrera F (2009) A survey on the application of genetic programming to classification. IEEE Trans Syst Man Cybern Part C (Appl Rev) 40(2):121–144

    Article  Google Scholar 

  32. Liu K, Xu C (2009) A genetic programming-based approach to the classification of multiclass microarray datasets. Bioinformatics 25(3):331–337

    Article  Google Scholar 

  33. Ma J, Gao X (2020) A filter-based feature construction and feature selection approach for classification using Genetic Programming. Knowl-Based Syst 196:105806

    Article  Google Scholar 

  34. Nag K, Pal NR (2020) Feature extraction and selection for parsimonious classifiers with multiobjective genetic programming. IEEE Trans Evol Comput 24(3):454–466

    Google Scholar 

  35. Neshatian K, Zhang M (2009) Pareto front feature selection: using genetic programming to explore feature space. In: Proceedings of the 11th annual conference on genetic and evolutionary computation

  36. Bi Y, Xue B, Zhang M (2020) An effective feature learning approach using genetic programming with image descriptors for image classification [research frontier]. IEEE Comput Intell Mag 15(2):65–77

    Article  Google Scholar 

  37. Fan Q, Xue B, Zhang M (2020) A region adaptive image classification approach using genetic programming. In: 2020 IEEE Congress on evolutionary computation (CEC)

  38. Ghazouani H (2021) A genetic programming-based feature selection and fusion for facial expression recognition. Appl Soft Comput 107173

  39. Barbiero P, et al (2019) A novel outlook on feature selection as a multi-objective problem. In: International conference on artificial evolution (evolution artificielle)

  40. Tubishat M et al (2020) Improved Salp Swarm algorithm based on opposition based learning and novel local search algorithm for feature selection. Exp Syst Appl 145:113122

    Article  Google Scholar 

  41. Abdel-Basset M et al (2020) A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection. Expert Syst Appl 139:112824

    Article  Google Scholar 

  42. Tran B, Xue B, Zhang M (2016) Genetic programming for feature construction and selection in classification on high-dimensional data. Memetic Comput 8(1):3–15

    Article  Google Scholar 

  43. Ingalalli V, et al. (2014) A multi-dimensional genetic programming approach for multi-class classification problems. In: European conference on genetic programming

  44. Chen Q, Zhang M, Xue B (2017) Feature selection to improve generalization of genetic programming for high-dimensional symbolic regression. IEEE Trans Evol Comput 21(5):792–806

    Article  Google Scholar 

  45. Li X, et al (2013) A spontaneous micro-expression database: inducement, collection and baseline. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (fg)

  46. Davison AK et al (2016) Samm: a spontaneous micro-facial movement dataset. IEEE Trans Affect Comput 9(1):116–129

    Article  Google Scholar 

  47. Liong S-T, et al (2015) Automatic apex frame spotting in micro-expression database. In: 2015 3rd IAPR Asian conference on pattern recognition (ACPR). IEEE

  48. Hunt R, Neshatian K, Zhang M (2012) A genetic programming approach to hyper-heuristic feature selection. In: Asia-Pacific conference on simulated evolution and learning

  49. Du H et al (2018) Elitism and distance strategy for selection of evolutionary algorithms. IEEE Access 6:44531–44541

    Article  Google Scholar 

  50. Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  51. Hu M et al (2019) Facial expression recognition using fusion features based on center-symmetric local octonary pattern. IEEE Access 7:29882–29890

    Article  Google Scholar 

  52. Päivärinta J, Rahtu E, Heikkilä J (2011) Volume local phase quantization for blur-insensitive dynamic texture classification. In: Scandinavian conference on image analysis, pp 360–369

  53. Qin X, Wang Z (2019) Nasnet: a neuron attention stage-by-stage net for single image deraining. arXiv preprint arXiiv:1912.03151

  54. Qassim H, Verma A, Feinzimer D (2018) Compressed residual-VGG16 CNN model for big data places image recognition. In: 2018 IEEE 8th annual computing and communication workshop and conference (CCWC)

  55. Carvalho T, et al (2017) Exposing computer generated images by eye’s region classification via transfer learning of VGG19 CNN. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA)

  56. Gan Y et al (2019) Off-apexnet on micro-expression recognition system. Signal Process Image Commun 74:129–139

    Article  Google Scholar 

  57. Sun L, Pan Q, Jing X et al (2021) A light-robust-optimization model and an effective memetic algorithm for an open vehicle routing problem under uncertain travel times. Memetic Comp 13:149–167

    Article  Google Scholar 

  58. Yang H, Meng C, Wang C (2021) A probability first memetic algorithm for the dynamic multiple-fault diagnosis problem with non-ideal tests. Memetic Comp 12:101–113

    Article  Google Scholar 

  59. Li G, Zhu Z, Ma L et al (2021) Multi-objective memetic algorithm for core-periphery structure detection in complex network. Memetic Comp 13:285–306

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61772023) and Natural Science Foundation of Fujian Province (No. 2016J01320).

Author information

Authors and Affiliations

  1. School of Informatics, Xiamen University, Xiamen, 361005, China

    Zhan WangPing, Jiang Min, Yao JunFeng, Liu KunHong & Wu QingQiang

Authors
  1. Zhan WangPing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiang Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yao JunFeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liu KunHong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wu QingQiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Liu KunHong or Wu QingQiang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

WangPing, Z., Min, J., JunFeng, Y. et al. The design of evolutionary feature selection operator for the micro-expression recognition. Memetic Comp. 14, 61–76 (2022). https://doi.org/10.1007/s12293-021-00350-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-021-00350-9

Keywords

Search

Navigation