Abstract
The machine learning-based classification model can predict the operator emotional states in human–machine system based on nonlinear, multidimensional neurophysiological features. However, the dynamical properties of the testing physiological data regarding the time series may influence the feature distribution variation and inter-class discrimination across different time steps. To overcome this shortcoming, we propose a novel EEG feature selection method, dynamical recursive feature elimination (D-RFE), to find the optimal but different feature rankings at each time instant for arousal and valence recognition. With the classification framework implemented via a model-selected least square support vector machine, the participant-specific classification performance has been significantly improved against conventional RFE model and several common classifiers. The optimal classification accuracy and F1-score elicited by the proposed method are 0.7896, 0.7991, 0.7143, and 0.7257 for arousal and valence dimensions, respectively, which are quite competitive among recent reported works on the same EEG database.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atkinson J, Campos D (2016) Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Syst Appl 47:35–41
Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F (2014) Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav R 44:58–75
Bravo ER, Ostos J (2017) Performance in computer-mediated work: the moderating role of level of automation. Cogn Technol Work 19:529–541
Brunner C, Vidaurre C, Billinger M, Neuper C (2011) A comparison of univariate, vector, bilinear autoregressive, and band power features for brain-computer interfaces. Med Biol Eng Comput 49:1337–1346
Chai X, Wang Q, Zhao Y, Liu X, Bai O, Li Y (2016) Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition. Comput Biol Med 79:205–214
Chen J, Hua B, Moore P, Zhang X, Ma X, (2015) Electroencephalogram-based emotion assessment system using ontology and data mining techniques. Appl Soft Comput 30:663–674
Degani A, Goldman CV, Deutsch O, Tsimhoni O (2017) On human–machine relations. Cogn Technol Work 19:211–231
Fanelli G, Gall J, Romsdorfer H, Weise T, Van Gool L (2010) A 3-D audio-visual corpus of affective communication. IEEE Trans Multimed 12:591–598
Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422
Hanjalic A, Xu L-Q (2005) Affective video content representation and modeling. IEEE Trans Multimed 7:143–154
Harbers M, Neerincx MA (2017) Value sensitive design of a virtual assistant for workload harmonization in teams. Cogn Technol Work 19:329–343
Huddlestone J, Harris D (2017) Doing more with fewer people: Human Factors contributions on the road to efficiency and productivity. Cogn Technol Work 19:207–209
Hyvarinen A (1999) Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans Neural Netw 10:626–634
Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014:627892
Kim J, Andre E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30:2067–2083
Koelstra S, Muehl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi TE, Pun T, Nijholt A, Patras IY (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3:18–31
Kramer A (1991) Physiological metrics of mental workload: a review of recent progress, multiple task performance. Taylor & Francis, Washington
Li X, Zhang P, Song D, Yu G, Hou Y, Hu B (2015) EEG based emotion identification using unsupervised deep feature learning. In: SIGIR2015 workshop on neuro-physiological methods in IR research, Santiago, Chile, 13 Aug 2015
Liu Y, Sourina O (2012) EEG-based valence level recognition for real-time applications. In: IEEE international conference on cyberworlds (CW), pp 53–60
Mehmood RM, Lee HJ (2016) A novel feature extraction method based on late positive potential for emotion recognition in human brain signal patterns. Comput Electr Eng 53:444–457
Naser DS, Saha G (2013) Recognition of emotions induced by music videos using DT-CWPT. In: Indian conference on medical informatics and telemedicine (ICMIT) IEEE, pp 53–57
Parasuraman R, Jiang Y (2012) Individual differences in cognition, affect, and performance: behavioral, neuroimaging, and molecular genetic approaches. NeuroImage 59:70–82
Schutte PC (2017) How to make the most of your human: design considerations for human–machine interactions. Cogn Technol Work 19:233–249
Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300
Ting C, Mahfouf M, Nassef A, Linkens D, Panoutsos G, Nickel P, Roberts A, Hockey GRJ (2010) Real-time adaptive automation system based on identification of operator functional state in simulated process control operations. IEEE Trans Syst Man Cybern A Syst Hum 40:251–262
Vapnik V (2000) The nature of statistical learning theory. Springer, New York
Verma GK, Tiwary US (2014) Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signal. NeuroImage 102:162–172
Wang D, Shang Y (2013) Modeling physiological data with deep belief networks. Int J Inf Educ Technol 3:505–511
Wang Z, Hope RM, Wang Z, Ji Q, Gray WD (2012) Cross-subject workload classification with a hierarchical Bayes model. Neuroimage 59:64–69
Yin Z, Zhang J (2014) Operator functional state classification using least-square support vector machine based recursive feature elimination technique. Comput Methods Prog Biomed 113:101–115
Yin Z, Zhao MY, Wang YX, Yang JD, Zhang J (2017a) Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Prog Biomed 140:93–110
Yin Z, Wang Y, Liu L, Zhang W, Zhang J (2017b) Cross-subject EEG feature selection for emotion recognition using transfer recursive feature elimination. Front Neurorobot 11:19
Yoon HJ, Chung SY (2013) EEG-based emotion estimation using Bayesian weighted-log-posterior function and perceptron convergence algorithm. Comput Biol Med 43:2230–2237
Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31:39–58
Zhang Q, Lee M (2013) Analyzing the dynamics of emotional scene sequence using recurrent neuro-fuzzy network. Cogn Neurodyn 7:47–57
Acknowledgements
This work was sponsored by The National Natural Science Foundation of China under Grant No. 61703277, The Shanghai Sailing Program (17YF1427000), The Shanghai Natural Science Fund (17ZR1419000), and The National Natural Science Foundation of China under Grant No. 61603256.
Author information
Authors and Affiliations
Contributions
ZY developed the D-RFE algorithm, performed the data analysis, and wrote the manuscript. LL, LL, and JZ advised data analysis and edited the original version of the manuscript. YW advised the revision of the method and the result sections.
Corresponding author
Rights and permissions
About this article
Cite this article
Yin, Z., Liu, L., Liu, L. et al. Dynamical recursive feature elimination technique for neurophysiological signal-based emotion recognition. Cogn Tech Work 19, 667–685 (2017). https://doi.org/10.1007/s10111-017-0450-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-017-0450-2