Abstract
In the context of emotion recognition, Artificial Intelligence technology has demonstrated several functions in people's lives. Computing research is now focused on Electroencephalogram (EEG) signals to identify emotional states. The connection and interaction between multichannel EEG signals give important information about emotional states. However, most existing emotion identification techniques perform poorly in practical applications by preventing their advancement. The main objective of this paper is to design an efficient model for emotion recognition based on deep learning technology by EEG signals. The proposed model for emotion recognition collects the EEG signals from the standard benchmark datasets. Then, the signal decomposition is performed using the Tunable Q-factor Wavelet Transform with the collected EEG signals. The decomposed signals are taken for the optimal feature selection phase, where the significant features of the emotion are selected through the hybrid optimization algorithm named Aquila Fireworks Optimization Algorithm (AFOA). Finally, the EEG emotion classification is performed using Hybrid Variational Autoencoder and Block Recurrent Transformer Network. The tuning of the parameter is made through the same AFOA to improve the efficiency of classification. The experimental analysis is conducted to analyze the efficiency of the recommended emotion recognition model with the comparison over the traditional techniques.












Similar content being viewed by others
Data availability
The data underlying this article are available in EEG Brainwave Dataset: Feeling Emotions, at https://www.kaggle.com/datasets/birdy654/eeg-brainwave-dataset-feeling-emotions: Access Date: 2023–01-19 and Emotion-Recognition-from-DEAP, at https://github.com/hi-akshat/Emotion-Recogniton-from-EEG-Signals: Access Date: 2023–07-19.
References
Shojaeilangari, S., Yau, W.-Y., Nandakumar, K., Li, J., Teoh, E.K.: Robust representation and recognition of facial emotions using extreme sparse learning. IEEE Trans. Image Process. 24(7), 2140–2152 (2015)
Wu, C.-H., Liang, W.-B.: Emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information and semantic labels. IEEE Trans. Affect. Comput. 2(1), 10–21 (2011)
Yoon, W.-J., Park, K.-S.: Building robust emotion recognition system on heterogeneous speech databases. IEEE Trans. Consum. Electron. 57(2), 747–750 (2011)
Zhang, C., Xue, L.: Autoencoder With emotion embedding for speech emotion recognition. IEEE Access 9, 51231–51241 (2021)
Yan, J., Zheng, W., Xu, Q., Lu, G., Li, H., Wang, B.: Sparse Kernel reduced-rank regression for bimodal emotion recognition from facial expression and speech. IEEE Trans. Multimedia 18(7), 1319–1329 (2016)
Huang, Y., Xiao, J., Tian, K., Wu, A., Zhang, G.: Research on robustness of emotion recognition under environmental noise conditions. IEEE Access 7, 142009–142021 (2019)
Zhang, H., Xu, M.: Weakly supervised emotion intensity prediction for recognition of emotions in images. IEEE Trans. Multimedia 23, 2033–2044 (2021)
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, Y.: Spatial–temporal recurrent neural network for emotion recognition. IEEE Trans. Cybernet. 49(3), 839–847 (2019)
Han, J., Zhang, Z., Ren, Z., Schuller, B.: EmoBed: strengthening monomodal emotion recognition via training with crossmodal emotion embeddings. IEEE Trans. Affect. Comput. 12(3), 553–564 (2021)
Cimtay, Y., Ekmekcioglu, E., Caglar-Ozhan, S.: Cross-subject multimodal emotion recognition based on hybrid fusion. IEEE Access 8, 168865–168878 (2020)
Yoon, Y.C.: Can we exploit all datasets? Multimodal emotion recognition using cross-modal translation. IEEE Access 10, 64516–64524 (2022)
Li, P., et al.: EEG based emotion recognition by combining functional connectivity network and local activations. IEEE Trans. Biomed. Eng. 66(10), 2869–2881 (2019)
Guo, J., et al.: Dominant and complementary emotion recognition from still images of faces. IEEE Access 6, 26391–26403 (2018)
Li, Y., Zheng, W., Zong, Y., Cui, Z., Zhang, T., Zhou, X.: A bi-hemisphere domain adversarial neural network model for EEG emotion recognition. IEEE Trans. Affect. Comput. 12(2), 494–504 (2021)
Ayari, N., Abdelkawy, H., Chibani, A., Amirat, Y.: Hybrid model-based emotion contextual recognition for cognitive assistance services. IEEE Trans. Cybernet. 52(5), 3567–3576 (2022)
Li, J., Qiu, S., Shen, Y.-Y., Liu, C.-L., He, H.: multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans. Cybernet. 50(7), 3281–3293 (2020)
Chen, L., Li, M., Su, W., Wu, M., Hirota, K., Pedrycz, W.: Adaptive feature selection-based AdaBoost-KNN with direct optimization for dynamic emotion recognition in human-robot interaction. IEEE Trans. Emerg. Top. Comput. Intell. 5(2), 205–213 (2021)
Chao, H., Dong, L.: Emotion recognition using three-dimensional feature and convolutional neural network from multichannel EEG signals. IEEE Sens. J. 21(2), 2024–2034 (2021)
Zhang, H.: Expression-EEG based collaborative multimodal emotion recognition using deep autoencoder. IEEE Access 8, 164130–164143 (2020)
Majid Mehmood, R., Du, R., Lee, H.J.: Optimal feature selection and deep learning ensembles method for emotion recognition from human brain EEG sensors. IEEE Access 5, 14797–14806 (2017)
Song, T., Zheng, W., Song, P., Cui, Z.: EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput. 11(3), 532–541 (2020)
Maheshwari, D., Ghosh, S.K., Tripathy, R.K., Sharma, M., RajendraAcharya, U.: Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals. Comput. Biol. Med. 134, 104428 (2021)
Zhong, M., Yang, Q., Liu, Y., Zhen, B., Zhao, F., Xie, B.: EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network. Biomed. Signal Process. Control 79(Part 2), 104211 (2023)
Cheng, W.X., Gao, R., Suganthan, P.N., Yuen, K.F.: EEG-based emotion recognition using random Convolutional Neural Networks. Eng. Appl. Arti. Intell. 116, 105349 (2022)
Bagherzadeh, S., Maghooli, K., Shalbaf, A., Maghsoudi, A.: Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals. Cognitive Neurodyn. 16, 1087–1106 (2022)
Abualigah, L., Yousri, D., Elaziz, M.A., Ewees, A.A., Al-qaness, M.A.A., Gandomi, A.H.: Aquila optimizer: a novel meta-heuristic optimization algorithm. Comput. Ind. Eng. 157, 107250 (2021)
Tan, Y., Zhu, Y.: Fireworks algorithm for optimization. Adv. Swarm Intell. First Int. Conf. 1, 355–364 (2010)
Hutchins, D., Schlag, I., Wu, Y., Dyer, E., Neyshabur, B.: Block-recurrent transformers. In: 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022)
Zhang, Y., Chi, A., Mirjalili, S.: Enhanced Jaya algorithm: a simple but efficient optimization method for constrained engineering design problems. Knowl.Based Syst. 233, 107555 (2021)
Abdollahzadeh, B., Gharehchopogh, F.S., Mirjalili, S.: Artificial gorilla troops optimizer: a new nature-inspired metaheuristic algorithm for global optimization problems. Int. J. Intell. Syst. 36, 434 (2021)
Liu, D., Liu, G.: A Transformer-based variational autoencoder for sentence generation. In: Conference: 2019 International Joint Conference on Neural Networks (IJCNN), July 2019 (2019)
Foggia, P., Greco, A., Saggese, A., Vento, M.: Multi-task learning on the edge for effective gender, age, ethnicity and emotion recognition. Eng. Appl. Artif. Intell. 118, 105651 (2023)
Mukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., Cho, J.: Masked face emotion recognition based on facial landmarks and deep learning approaches for visually impaired people. Sensors 23(3), 1080 (2023)
Sams, A.S., Zahra, A.: Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers. Bull. Electrical Eng. Inform. 12(1), 355–364 (2023)
Bilal Er, M., Çiğ, H., Aydilek, İB.: A new approach to recognition of human emotions using brain signals and music stimuli. Appl. Acoust. 175, 107840 (2021)
Krishna, A.H., Sri, A.B., Priyanka, K.Y.V.S., Taran, S., Bajaj, V.: Emotion classification using EEG signals based on tunable-Q wavelet transform. IET Sci. Measure. Technol. 13(3), 375–380 (2019)
Issa, S., Peng, Q., You, X.: Emotion classification using EEG brain signals and the broad learning system. IEEE Trans. Syst. Man Cybernet. Syst. 51(12), 7382–7391 (2021)
Issa, H., Peng, Q., Issa, S., You, X., Peng, R., Wang, J.: Person-independent emotion and gender prediction (EGP) system using EEG signals. Int. Arab J. Inf. Technol. 19, 4 (2022)
Funding
This research did not receive any specific funding.
Author information
Authors and Affiliations
Contributions
All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Reddy, C.H.N., Mahesh, S. & Manjunathachari, K. Intelligent optimal feature selection-based hybrid variational autoencoder and block recurrent transformer network for accurate emotion recognition model using EEG signals. SIViP 18, 1027–1039 (2024). https://doi.org/10.1007/s11760-023-02702-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-023-02702-z