Skip to main content
SpringerLink
Log in

Multimodal biometric authentication based on deep fusion of electrocardiogram (ECG) and finger vein

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Biometric identification depends on the statistical analysis of the unique physical and behavioral characteristics of individuals. However, a unimodal biometric system is susceptible to different attacks such as spoof attacks. To overcome these limitations, we propose a multimodal biometric authentication system based on deep fusion of electrocardiogram (ECG) and finger vein. The proposed system has three main components, which are biometric pre-processing, deep feature extraction, and authentication. During the pre-processing, normalization and filtering techniques are adapted for each biometric. In the feature extraction process, the features are extracted using a proposed deep Convolutional Neural Network (CNN) model. Then, the authentication process is performed on the extracted features using five well-known machine learning classifiers: Support Vector Machine (SVM), K-Nearest Neighbors (KNNs), Random Forest (RF), Naive Bayes (NB), and Artificial Neural Network (ANN). In addition, to represent the deep features in a low-dimensional feature space and speed up the authentication task, we adopt Multi-Canonical Correlation Analysis (MCCA). We combine the two biometric systems based on ECG and finger vein into a single multimodal biometric system using feature and score fusion. The performance of the proposed system is tested on two finger vein (TW finger vein and VeinPolyU finger vein) databases and two ECG (MWM-HIT and ECG-ID) databases. Experimental results reveal improvement in terms of authentication performance with Equal Error Rates (EERs) of 0.12% and 1.40% using feature fusion and score fusion, respectively. Furthermore, the authentication with the proposed multimodal system using MCCA feature fusion with a KNN classifier shows an increase of accuracy by an average of 10% compared with those of other machine learning algorithms. Therefore, the proposed biometric system is effective in performing secure authentication and assisting the stakeholders in making accurate authentication of users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Sajjad, M., Khan, S., Hussain, T., Muhammad, K., Sangaiah, A.K., Castiglione, A., Baik, S.W.: CNN-based anti-spoofing two-tier multi-factor authentication system. Pattern Recogn. Lett. 126, 123–131 (2019)

    Article  Google Scholar 

  2. Herbadji, A., Akhtar, Z., Siddique, K., Guermat, N., Ziet, L., Cheniti, M., Muhammad, K.: Combining multiple biometric traits using asymmetric aggregation operators for improved person recognition. Symmetry 12(3), 444 (2020)

    Article  Google Scholar 

  3. Fadl, S., Megahed, A., Han, Q., Qiong, L.: Frame duplication and shuffling forgery detection technique in surveillance videos based on temporal average and gray level co-occurrence matrix. Multimedia Tools Appl. 79, 17619–17643 (2020)

    Article  Google Scholar 

  4. Abd El-Latif, A.A., Hossain, M.S., Wang, N.: Score level multibiometrics fusion approach for healthcare. Clust. Comput. 22(1), 2425–2436 (2019)

    Article  Google Scholar 

  5. Ali, Z., Hossain, M.S., Muhammad, G., Ullah, I., Abachi, H., Alamri, A.: Edge-centric multimodal authentication system using encrypted biometric templates. Futur. Gener. Comput. Syst. 85, 76–87 (2018)

    Article  Google Scholar 

  6. Abdul, W., Ali, Z., Ghouzali, S., Alfawaz, B., Muhammad, G., Hossain, M.S.: Biometric security through visual encryption for fog edge computing. IEEE Access 5, 5531–5538 (2017)

    Article  Google Scholar 

  7. Xiaoding, W., Garg, S., Lin, H., Jalilpiran, M., Hu, J., & Hossain, M. S. (2021). Enabling secure authentication in industrial IoT with transfer learning empowered Blockchain. IEEE Trans. Ind. Inf.

  8. Hossain, M.S., Muhammad, G., Qurishi, M.A.: Verifying the images authenticity in cognitive internet of things (CIoT)-oriented cyber physical system. Mob. Netw. Appl. 23(2), 239–250 (2018)

    Article  Google Scholar 

  9. Gad, R., Talha, M., El-Latif, A.A.A., Zorkany, M., Ayman, E.S., Nawal, E.F., Muhammad, G.: Iris recognition using multi-algorithmic approaches for cognitive internet of things (CIoT) framework. Futur. Gener. Comput. Syst. 89, 178–191 (2018)

    Article  Google Scholar 

  10. Wang, N., Li, Q., El-Latif, A.A.A., Peng, J., Niu, X.: An enhanced thermal face recognition method based on multiscale complex fusion for Gabor coefficients. Multimedia Tools Appl. 72(3), 2339–2358 (2014)

    Article  Google Scholar 

  11. Wang, N., Li, Q., El-Latif, A.A.A., Zhang, T., Niu, X.: Toward accurate localization and high recognition performance for noisy iris images. Multimedia Tools Appl. 71(3), 1411–1430 (2014)

    Article  Google Scholar 

  12. Hammad, M., & Wang, K.: Fingerprint classification based on a Q-Gaussian multiclass support vector machine. In: Proceedings of the 2017 International Conference on Biometrics Engineering and Application, pp. 39–44 (2017)

  13. Omara, I., Emam, M., Hammad, M., Zuo, W.: Ear verification based on a novel local feature extraction. In: Proceedings of the 2017 International Conference on Biometrics Engineering and Application, pp. 28–32 (2017)

  14. Singh, M., Singh, R., Ross, A.: A comprehensive overview of biometric fusion. Inf. Fusion 52, 187–205 (2019)

    Article  Google Scholar 

  15. Peng, J., Li, Q., El-Latif, A.A.A., Wang, N., Niu, X.: Finger vein recognition with gabor wavelets and local binary patterns. IEICE Trans. Inf. Syst. 96(8), 1886–1889 (2013)

    Article  Google Scholar 

  16. Alghamdi, A., Hammad, M., Ugail, H., Abdel-Raheem, A., Muhammad, K., Khalifa, H.S., Abd El-Latif, A.A.: Detection of myocardial infarction based on novel deep transfer learning methods for urban healthcare in smart cities. Multimedia Tools Appl. 1–22 (2020)

  17. Hammad, M., Maher, A., Wang, K., Jiang, F., Amrani, M.: Detection of abnormal heart conditions based on characteristics of ECG signals. Measurement 125, 634–644 (2018)

    Article  Google Scholar 

  18. Amrani, M., Hammad, M., Jiang, F., Wang, K., Amrani, A.: Very deep feature extraction and fusion for arrhythmias detection. Neural Comput. Appl. 30(7), 2047–2057 (2018)

    Article  Google Scholar 

  19. Hammad, M., Iliyasu, A.M., Subasi, A., Ho, E.S., Abd El-Latif, A.A.: A multitier deep learning model for arrhythmia detection. IEEE Trans. Instrum. Meas. 70, 1–9 (2020)

    Article  Google Scholar 

  20. Hammad, M., Alkinani, M.H., Gupta, B.B., Abd El-Latif, A.A.: Myocardial infarction detection based on deep neural network on imbalanced data. Multimedia Syst. 1–13 (2021)

  21. Hammad, M., Pławiak, P., Wang, K., Acharya, U.R.: ResNet‐Attention model for human authentication using ECG signals. Expert Syst. e12547 (2020)

  22. Hammad, M., Luo, G., Wang, K.: Cancelable biometric authentication system based on ECG. Multimedia Tools Appl. 78(2), 1857–1887 (2019)

    Article  Google Scholar 

  23. Hammad, M., Zhang, S., Wang, K.: A novel two-dimensional ECG feature extraction and classification algorithm based on convolution neural network for human authentication. Futur. Gener. Comput. Syst. 101, 180–196 (2019)

    Article  Google Scholar 

  24. Hammad, M., Ibrahim, M., Hadhoud, M.: A novel biometric based on ECG signals and images for human authentication. Int. Arab J. Inf. Technol. 13(6A), 959–964 (2016)

    Google Scholar 

  25. Su, K., Yang, G., Wu, B., Yang, L., Li, D., Su, P., Yin, Y.: Human identification using finger vein and ECG signals. Neurocomputing 332, 111–118 (2019)

    Article  Google Scholar 

  26. Lu, Y., Xie, S.J., Yoon, S., Wang, Z., Park, D.S.: An available database for the research of finger vein recognition. In: 2013 6th International Congress on Image and Signal Processing (CISP), Vol. 1, pp. 410–415. IEEE (2013)

  27. Kumar, A., Zhou, Y.: Human identification using finger images. IEEE Trans. Image Process. 21(4), 2228–2244 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  28. Lugovaya T.S. Biometric human identification based on electrocardiogram. [Master's thesis] Faculty of Computing Technologies and Informatics, Electrotechnical University “LETI”, Saint-Petersburg, Russian Federation (2005)

  29. Goldberger, A.L., Amaral, L.A.N., Glass, L., Hausdorff, J.M., Ivanov, PCh., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.-K., Stanley, H.E.: PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. 101(23), e215–e220 (2003)

    Google Scholar 

  30. Yang, W., Wang, S., Hu, J., Zheng, G., Yang, J., Valli, C.: Securing deep learning based edge finger vein biometrics with binary decision diagram. IEEE Trans. Industr. Inf. 15(7), 4244–4253 (2019)

    Article  Google Scholar 

  31. Hu, N., Ma, H., Zhan, T.: Finger vein biometric verification using block multi-scale uniform local binary pattern features and block two-directional two-dimension principal component analysis. Optik 163664 (2020)

  32. Liu, C.Y., Ruan, S.J., Lai, Y.R., Yao, C.Y.: Finger-vein as a biometric-based authentication. IEEE Consumer Electron. Mag. 8(6), 29–34 (2019)

    Article  Google Scholar 

  33. Peng, J., Li, Q., Wang, N., El-Latif, A.A.A., Niu, X.: An effective preprocessing method for finger vein recognition. In Fifth International Conference on Digital Image Processing (ICDIP 2013), Vol. 8878, p. 887808. International Society for Optics and Photonics (2013)

  34. Peng, J., Wang, N., El-Latif, A.A.A., Li, Q., Niu, X.: Finger-vein verification using Gabor filter and sift feature matching. In: 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 45–48. IEEE (2012)

  35. Matsuda, Y., Miura, N., Nagasaka, A., Kiyomizu, H., Miyatake, T.: Finger-vein authentication based on deformation-tolerant feature-point matching. Mach. Vis. Appl. 27(2), 237–250 (2016)

    Article  Google Scholar 

  36. Peng, J., Li, Q., El-Latif, A.A.A., Niu, X.: Finger multibiometric cryptosystem based on score-level fusion. Int. J. Comput. Appl. Technol. 51(2), 120–130 (2015)

    Article  Google Scholar 

  37. Yang, J., Zhang, X.: Feature-level fusion of fingerprint and finger-vein for personal identification. Pattern Recogn. Lett. 33(5), 623–628 (2012)

    Article  Google Scholar 

  38. Peng, J., El-Latif, A.A.A., Li, Q., Niu, X.: Multimodal biometric authentication based on score level fusion of finger biometrics. Optik 125(23), 6891–6897 (2014)

    Article  Google Scholar 

  39. Kirchgasser, S., Kauba, C., Lai, Y.L., Zhe, J., Uhl, A.: Finger vein template protection based on alignment-robust feature description and index-of-maximum hashing. IEEE Trans. Biometr. Behav. Identity Sci. 2(4), 337–349 (2020)

    Article  Google Scholar 

  40. Kauba, C., Prommegger, B., Uhl, A.: The two sides of the finger-an evaluation on the recognition performance of dorsal vs. palmar finger-veins. In: 2018 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1–5. IEEE (2018)

  41. Hu, N., Ma, H., Zhan, T.: Finger vein biometric verification using block multi-scale uniform local binary pattern features and block two-directional two-dimension principal component analysis. Optik 208, 163664 (2020)

    Article  Google Scholar 

  42. Meng, X., Xi, X., Li, Z., Zhang, Q.: Finger Vein recognition based on fusion of deformation information. IEEE Access 8, 50519–50530 (2020)

    Article  Google Scholar 

  43. Yang, W., Luo, W., Kang, W., Huang, Z., Wu, Q.: FVRAS-Net: an embedded finger-vein recognition and AntiSpoofing system using a unified CNN. IEEE Trans. Instrum. Meas. 69(11), 8690–8701 (2020)

    Article  Google Scholar 

  44. Zhao, Z., Zhang, Y., Deng, Y., Zhang, X.: ECG authentication system design incorporating a convolutional neural network and generalized S-Transformation. Comput. Biol. Med. 102, 168–179 (2018)

    Article  Google Scholar 

  45. Peris-Lopez, P., González-Manzano, L., Camara, C., de Fuentes, J.M.: Effect of attacker characterization in ECG-based continuous authentication mechanisms for Internet of Things. Futur. Gener. Comput. Syst. 81, 67–77 (2018)

    Article  Google Scholar 

  46. Huang, P., Guo, L., Li, M., Fang, Y.: Practical Privacy-preserving ECG-based authentication for IoT-based Healthcare. IEEE Internet Things J. 6(5), 9200–9210 (2019)

    Article  Google Scholar 

  47. Chu, Y., Shen, H., Huang, K.: ECG authentication method based on parallel multi-scale one-dimensional residual network with center and margin loss. IEEE Access 7, 51598–51607 (2019)

    Article  Google Scholar 

  48. Arteaga-Falconi, J.S., Al Osman, H., El Saddik, A.: ECG and fingerprint bimodal authentication. Sustain. Cities Soc. 40, 274–283 (2018)

    Article  Google Scholar 

  49. Hsiao, C.-C., Wang, S.-W., Lin, R., Lee, R.-G.: Multiple biometric authentication for personal identity using wearable device. In: Systems, Man, and Cybernetics (SMC), 2016 IEEE International Conference on, pp. 000673-000678 (2016)

  50. Hammad, M., Liu, Y., Wang, K.: Multimodal biometric authentication systems using convolution neural network based on different level fusion of ECG and fingerprint. IEEE Access 7, 26527–26542 (2018)

    Article  Google Scholar 

  51. Hammad, M., Wang, K.: Parallel score fusion of ECG and fingerprint for human authentication based on convolution neural network. Comput. Secur. 81, 107–122 (2019)

    Article  Google Scholar 

  52. Yang, L., Yang, G., Yin, Y., Xiao, R.: Sliding window-based region of interest extraction for finger vein images. Sensors 13(3), 3799–3815 (2013)

    Article  Google Scholar 

  53. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  54. Haghighat, M., Abdel-Mottaleb, M., Alhalabi, W.: Fully automatic face normalization and single sample face recognition in unconstrained environments. Expert Syst Appl 47, 23–34 (2016)

    Article  Google Scholar 

  55. Liu, C., Wechsler, H.: A shape- and texture-based enhanced Fisher classifier for face recognition. IEEE Trans. Image Process. 10(4), 598–608 (2001)

    Article  MATH  Google Scholar 

  56. Fierrez, J., Morales, A., Vera-Rodriguez, R., Camacho, D.: Multiple classifiers in biometrics. Part 1: fundamentals and review. Inf. Fusion 44, 57–64 (2018)

    Article  Google Scholar 

  57. Chaib, S., Liu, H., Gu, Y., Yao, H.: Deep feature fusion for VHR remote sensing scene classification. IEEE Trans. Geosci. Remote Sens. 55(8), 4775–4784 (2017)

    Article  Google Scholar 

  58. Modak, S.K.S., Jha, V.K.: Multibiometric fusion strategy and its applications: a review. Inf. Fusion 49, 174–204 (2019)

    Article  Google Scholar 

  59. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  60. Aburomman, A.A., Reaz, M.B.I.: A novel SVM-kNN-PSO ensemble method for intrusion detection system. Appl. Soft Comput. 38, 360–372 (2016)

    Article  Google Scholar 

  61. Dureja, H., Gupta, S., Madan, A.K.: Topological models for prediction of pharmacokinetic parameters of cephalosporins using random forest, decision tree and moving average analysis. Sci. Pharm. 76(3), 377–394 (2008)

    Article  Google Scholar 

  62. Perdana, R.S., Pinandito, A.: Combining likes-retweet analysis and naive Bayes classifier within twitter for sentiment analysis. J. Telecommun. Electron. Comput. Eng. (JTEC) 10(1–8), 41–46 (2018)

    Google Scholar 

  63. Tadeusiewicz, R.: Neural networks in mining sciences–general overview and some representative examples. Arch. Min. Sci. 60(4), 971–984 (2015)

    Google Scholar 

  64. Veluchamy, S., Karlmarx, L.R.: System for multimodal biometric recognition based on finger knuckle and finger vein using feature-level fusion and k-support vector machine classifier. IET Biometr. 6(3), 232–242 (2016)

    Article  Google Scholar 

  65. Mishra, D., Naik, B., Sahoo, R.M., Nayak, J.: Deep recurrent neural network (Deep-RNN) for classification of nonlinear data. In: Computational Intelligence in Pattern Recognition, pp. 207–215. Springer, Singapore (2020)

  66. Nayak, J., Naik, B.: A novel honey-bees mating optimization approach with higher order neural network for classification. J. Classif. 35(3), 511–548 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  67. Nayak, J., Vakula, K., Dinesh, P., Naik, B.: Applications and advancements of firefly algorithm in classification: an analytical perspective. In: Computational Intelligence in Pattern Recognition, pp. 1011–1028. Springer, Singapore (2020)

Download references

Author information

Authors and Affiliations

  1. Mathematics and Computer Science Department, Faculty of Science, Menoufia University, Shebin El-Koom, Egypt

    Basma Abd El-Rahiem & Mohamed Amin

  2. Department of Electronics and Electrical Communications Engineering, Faculty of Electronic Engineering, Menoufia University, Menouf, 32952, Egypt

    Fathi E. Abd El-Samie

Authors
  1. Basma Abd El-Rahiem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fathi E. Abd El-Samie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Amin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Basma Abd El-Rahiem.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El-Rahiem, B.A., El-Samie, F.E.A. & Amin, M. Multimodal biometric authentication based on deep fusion of electrocardiogram (ECG) and finger vein. Multimedia Systems 28, 1325–1337 (2022). https://doi.org/10.1007/s00530-021-00810-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-021-00810-9

Keywords

Search

Navigation