Skip to main content
SpringerLink
Log in

Multimodal biometric systems based on different fusion levels of ECG and fingerprint using different classifiers

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Multimodal biometric system can be accomplished at different levels of fusion and achieve higher recognition performance than the unimodal system. This paper concerned to study the performance of different classification techniques and fusion rules in the context of unimodal and multimodal biometric systems based on the electrocardiogram (ECG) and fingerprint. The experiments are conducted on ECG and fingerprint databases to evaluate the performance of the proposed biometric systems. MIT-BIH database is utilized for ECG, FVC2004 database is utilized for the fingerprint, and further experiments are being performed to evaluate the proposed multimodal system with 47 subjects from virtual multimodal database. The performance of the proposed unimodal and multimodal biometric systems is measured using receiver operating characteristic (ROC) curve, AUC (area under the ROC curve), sensitivity, specificity, efficiency, standard error of the mean, and likelihood ratio. The findings indicate AUC up to 0.985 for sequential multimodal system, and up to 0.956 for parallel multimodal system, as compared to the unimodal systems that achieved AUC up to 0.951, and 0.866, for the ECG and fingerprint biometrics, respectively. The overall performance of the proposed multimodal systems is better than that of the unimodal systems based on different classifiers and different fusion levels and rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Acharya UR, Oh SL, Hagiwara Y, Tan JH, Adam M, Gertych A, Tan RS (2017) A deep convolutional neural network model to classify heartbeats. J Comput Biol Med 89:389–396

    Google Scholar 

  • Ali MMH, Mahale VH, Yannawar P, Gaikwad AT (2016) Overview of fingerprint recognition system. In: International conference on electrical, electronics, and optimization techniques (ICEEOT)-©2016 IEEE, Chennai, India. https://doi.org/10.1109/iceeot.2016.7754900

  • Arthur J (2019) QI Macros. ©2019 KnowWare International Inc. https://www.qimacros.com/qi-macros/. Accessed 5 Dec 2019

  • Barde MP, Barde PJ (2012) What to use to express the variability of data: standard deviation or standard error of mean? Perspect Clin Res 3(3):113–116. https://doi.org/10.4103/2229-3485.100662

    Article  Google Scholar 

  • Barra S, Casanova A, Fraschini M, Nappi M, Barra S, Casanova A, Fraschini M, Nappi M (2017) Fusion of physiological measures for multimodal biometric systems. Multimed Tools Appl 76(4):4835–4847. https://doi.org/10.1007/s11042-016-3796-1

    Article  Google Scholar 

  • Belgacem N, Nait-Ali A, Fournier R, Bereksi-Reguig F (2012) ECG based human authentication using wavelets and random forests. Int J Cryptogr Inf Secur (IJCIS) 2(2):1–11

    Google Scholar 

  • Berrar D (2017) Confidence curves: an alternative to null hypothesis significance testing for the comparison of classifiers. Mach Learn 106:911–949. https://doi.org/10.1007/s10994-016-5612-6

    Article  MathSciNet  MATH  Google Scholar 

  • Bhargava N, Bhargava R, Mathuria M, Cotia M (2012) Fingerprint matching using ridge-end and bifurcation points. In: IJCA proceedings on international conference on recent trends in information technology and computer science, 2012 ICRTITCS 6, pp 12–15

  • Biel L, Pettersson O, Philipson L, Wide P (2001) ECG analysis: a new approach in human identification. IEEE Trans Instrum Meas 50(3):808–812

    Google Scholar 

  • Borra SR, Reddy GJ, Reddy ES (2018) Classification of fingerprint images with the aid of morphological operation and AGNN classifier. Appl Comput Inform 14:166–176

    Google Scholar 

  • Boumbarov O, Velchev Y, Tonchev K, Paliy I (2011) Face and ECG based multi-modal biometric authentication, advanced biometric technologies. In: Chetty G, Yang J (eds) InTechOpen. https://doi.org/10.5772/21842

  • Chen K, Wang L, Chi H (1997) Methods of combining multiple classifiers with different features and their applications to text-independent speaker identification. Int J Pattern Recognit Artif Intell 11(3):417–445

    Google Scholar 

  • Ching HC (2013) Performance evaluation of multimodal biometric systems using fusion techniques. PhD Thesis. Faculty of Computing and Informatics. Multimedia University. Malaysia

  • Dale MP, Joshi MA, Sahu MK (2007) DCT feature based fingerprint recognition. In: International conference on intelligent and advanced systems, Kuala Lumpur, Malaysia. https://doi.org/10.1109/icias.2007.4658460

  • Davis J, Goadrich M (2006) The relationship between precision-recall and ROC curves. In: ICML ‘06: proceedings of the 23rd international conference on machine learning. Pittsburgh, PA, ICML ‘06, pp 233–240

  • Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evolut Comput 1:3–18

    Google Scholar 

  • Duda RO, Hart PE, Stork DG (2000) Pattern classification, 2nd edn. Wiley, New York. ISBN 0471056693

  • El_Rahman SA (2019) biometric human recognition system based on ECG. Multimed Tools Appl 78:17555–17572. https://doi.org/10.1007/s11042-019-7152-0

    Article  Google Scholar 

  • Espiritu JD, Rolluqui G, Gustilo RC (2015) Neural network based partial fingerprint recognition as support for forensics. In: 8th IEEE international conference humanoid, nanotechnology, information technology, communication and control, environment and management (HNICEM), Cebu City, Philippines. https://doi.org/10.1109/hnicem.2015.7393227

  • Fatemian SZ (2009) A wavelet-based approach to electrocardiogram (ECG) and phonocardiogram (PCG) subject recognition. A thesis, Master of Applied Science, Graduate Department of Electrical and Computer Engineering, University of Toronto

  • Fung YH, Chan YH (2004) Fingerprint recognition with improved wavelet domain features. In: Proceedings of international intelligent multimedia, video and speech processing

  • FVC (2004) FVC2004: the third international fingerprint verification competition. http://bias.csr.unibo.it/fvc2004/download.asp. Accessed 31 June 2018

  • Guo J, Sun Z, Tang H, Jia X, Wang S, Yan X, Ye G, Wu G (2016) Hybrid optimization algorithm of particle swarm optimization and cuckoo search for preventive maintenance period optimization. Discrete Dynamics in Nature and Society. 2016(ID 1516271), p 12

  • Hammad M, Wang K (2019) Parallel score fusion of ECG and fingerprint for human authentication based on convolution neural network. J Comput Secur 81:107–122

    Google Scholar 

  • Hammad M, Liu Y, Wang K (2018) Multimodal biometric authentication systems using convolution neural network based on different level fusion of ECG and fingerprint. IEEE Access 6:26527–26542. https://doi.org/10.1109/access.2018.2886573

    Article  Google Scholar 

  • Hariyanto, Sudiro SA, Lukman S (2015) Minutiae matching algorithm using artificial neural network for fingerprint recognition. In: 2015 Third international conference on artificial intelligence, modelling and simulation, Kota Kinabalu, Malaysia, AIMS, pp 37–41. https://doi.org/10.1109/aims.2015.16

  • Ho TK, Hull JJ, Srihari SN (1994) Decision combination in multiple classifier systems. IEEE Trans Pattern Anal Mach Intell 16(1):66–75

    Google Scholar 

  • Hong L, Jain A, Pankanti SH (1999) Can multibiometrics improve performance? In: Proceedings of IEEE workshop on automatic identification advanced technologies, Proceedings AutoID’99, Summit(NJ), USA, pp 59–64

  • Hong WC, Li MW, Geng J, Zhang Y (2019) Novel chaotic bat algorithm for forecasting complex motion of floating platforms. Appl Math Model 72:425–443

    MathSciNet  MATH  Google Scholar 

  • Jacob AJ, Bhuvan NT, Thampi SM (2011) Feature level fusion using multiple fingerprints. In: IJCA special issue on computational science-new dimensions and perspectives, NCCSE (1), pp 13–18

  • Jaina A, Nandakumara K, Ross A (2005) Score normalization in multimodal biometric systems. J Pattern Recognit Soc 38:2270–2285

    Google Scholar 

  • Jeyanthi S, Maheswaria NU, Venkateshb R (2015) Neural network based automatic fingerprint recognition system for overlapped latent images. J Intell Fuzzy Syst 28(6):2889–2899

    Google Scholar 

  • Jha AK, Narasimham S, Krishna SS, Pillah VPM (2010) A neural network based approach for fingerprint recognition system. In: International congress on ultra modern telecommunications and control systems and workshops (ICUMT), Moscow, Russia, pp 808–812. https://doi.org/10.1109/icumt.2010.5676543

  • Kahraman N, Taskiran ZGC, Taskiran M (2018) Novel feature extraction methodology with evaluation in artificial neural networks based fingerprint recognition system. Tech Gaz 25(1):112–119. https://doi.org/10.17559/TV-20170816124949

    Article  Google Scholar 

  • Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239

    Google Scholar 

  • Kittler J, Li Y, Matas J (2000) On matching scores for LDA-based face verification. In: Proceedings of the British machine vision conference 2000, BMVC 2000, Bristol, UK, pp 42–51. https://doi.org/10.5244/c.14.5

  • Kondekar M, Kulkarni U, Chowhan S (2011) A fuzzy neural clustering approach for fingerprint recognition. IJCSI Int J Comput Sci 8(5):344–348

    Google Scholar 

  • Kundra H, Sadawarti H (2015) Hybrid algorithm of cuckoo search and particle swarm optimization for natural terrain feature extraction. Res J Inf Technol 7(1):58–69

    Google Scholar 

  • Lam L, Suen CY (1997) Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern Part A Syst Hum 27(5):553–568

    Google Scholar 

  • Lee DK (2017) Alternatives to P value: confidence interval and effect size. Korean J Anesthesiol 69(6):555–562. https://doi.org/10.4097/kjae.2016.69.6.555

    Article  Google Scholar 

  • Lee SW, Nam BH (1999) Fingerprint recognition using wavelet transform and probabilistic neural network. In: International joint conference on neural networks, IJCNN’99 Proceedings (Cat. No.99CH36339), Washington, DC, USA. https://doi.org/10.1109/ijcnn.1999.836183

  • Lee CJ, Wang SD (1999) A Gabor filter-based approach to fingerprint recognition. In: IEEE workshop on signal processing systems. SiPS 99. Design and implementation (Cat. No. 99TH8461), Taipei, Taiwan. https://doi.org/10.1109/sips.1999.822342

  • Liu B, Liu J, Wang G, Huang K, Li F, Zheng Y, Luo Y, Zhou F (2015) A novel electrocardiogram parameterization algorithm and its application in myocardial infarction detection. J Comput Biol Med 61:178–184

    Google Scholar 

  • Liwen FS, Cai XA, Ma J (2010) A dual-biometric-modality identification system based on fingerprint and EEG. In: Fourth IEEE international conference on biometrics: theory, applications and systems (BTAS), Washington, DC, USA. https://doi.org/10.1109/btas.2010.5634487

  • Manjunathswamy BE, Abhishek MA, Thriveni J, Venugopal KR, Patnaik LM (2014) Multimodal biometrics using ECG and fingerprint. IN: Proceedings of international conference on advances in communication network and computing, CNC, Chennai, India, pp 777–784

  • McGee S (2002) Simplifying likelihood ratios. J General Intern Med 17(8):647–650. https://doi.org/10.1046/j.1525-1497.2002.10750.x

    Article  Google Scholar 

  • Moody GB, Mark RG (2001) The impact of the MIT-BIH arrhythmia database. IEEE Eng Med Biol 20(3):45–50

    Google Scholar 

  • Mote P, Zope PH (2012) Multimodal biometric system using Gabor filter. Int J Adv Trends Comput Sci Eng 1(2):67–72

    Google Scholar 

  • Naderpour H, Mirrashid M (2019) Classification of failure modes in ductile and non-ductile concrete joints. Eng Fail Anal 103:361–375

    Google Scholar 

  • Nawal M, Purohit GN (2014) ECG based human authentication: a review. Int J Emerg Eng Res Technol 2(3):178–185

    Google Scholar 

  • Oulhiq R, Ibntahir S, Sebgui M, Guennoun Z (2015) A fingerprint recognition framework using artificial neural network. In: 2015 10th international conference on intelligent systems: theories and applications (SITA), Rabat, Morocco

  • Pal A, Singh YN (2018) ECG biometric recognition. In: Ghosh D, Giri D, Mohapatra R, Savas E, Sakurai K, Singh L (eds) Mathematics and computing. ICMC 2018. Communications in computer and information science, vol 834. Springer, Berlin, pp 61–73

    Google Scholar 

  • Pan J, Tompkins WJ (1985) A real-time QRS detection algorithm. IEEE Trans Biomed Eng BME 32(3):230–236

    Google Scholar 

  • Pant T, Han C, Wang H (2019) Examination of errors of table integration in flamelet/progress variable modeling of a turbulent non-premixed jet flame. Appl Math Model 72:369–384

    MathSciNet  MATH  Google Scholar 

  • Pouryayevali SH (2015) ECG biometrics: new algorithm and multimodal biometric system. A Thesis, Master of Applied Science, Department of Electrical and Computer Engineering, University of Toronto

  • Raju AS, Udayashankara V (2017) Database evaluation of ECG fingerprint and face multimodal biometric system. In: International conference on signal, image processing communication and automation (ICSIPCA), JSSATE, Bengaluru, 207-215. Grenze ID: 02.MH-ICSIPCA.2017.1.32

  • Ranstam J (2012) Why the P-value culture is bad and confidence intervals a better alternative. Osteoarthr Cartil 20(8):805–808. https://doi.org/10.1016/j.joca.2012.04.001

    Article  Google Scholar 

  • Roli F, Kittler J, Fumera G, Muntoni D (2002) An experimental comparison of classifier fusion rules for multimodal personal identity verification systems. In: Roli F, Kittler J (eds) Multiple classifier systems. International workshop on multiple classifier systems, MCS 2002. Lecture notes in computer science, vol 2364. Springer, Berlin, pp 325–335. https://doi.org/10.1007/3-540-45428-4_32

  • Sabri M, Moin MS, Razzazi F (2019) A new framework for match on card and match on host quality based multimodal biometric authentication. J Signal Process Syst 91:163–177. https://doi.org/10.1007/s11265-018-1385-4

    Article  Google Scholar 

  • Safdarian N, Dabanloo NJ, Attarodi G (2014) New pattern recognition method for detection and localization of myocardial infarction using t-wave integral and total integral as extracted features from one cycle of ECG signal. J Biomed Sci Eng 7(10):818–824

    Google Scholar 

  • Sanderson C, Paliwal KK (2004) Identity verification using speech and face information. Digital Signal Process 14(5):449–480. https://doi.org/10.1016/j.dsp.2004.05.001

    Article  Google Scholar 

  • SankaraNarayanan R, Shyamala K, Nagarajan M (2015) authoritative cropped fingerprint recognition using termination and bifurcation. Int J Eng Dev Res (IJEDR) 3(2):1–8

    Google Scholar 

  • Shekhar S, Patel VM, Nasrabadi NM, Chellappa R (2014) Joint sparse representation for robust multimodal biometrics recognition. IEEE Trans Pattern Anal Mach Intell 36(1):113–126

    Google Scholar 

  • Sim T, Zhang S, Janakiraman R, Kumar S (2007) Continuous verification using multimodal biometrics. IEEE Trans Pattern Anal Mach Intell 29(4):687–700

    Google Scholar 

  • Singh YN (2014) Individual identification using linear projection of heartbeat features. J Appl Comput Intell Soft Comput 2014:1–14

    Google Scholar 

  • Singh YN, Gupta P (2008) ECG to individual identification. In: 2nd IEEE international conference on biometrics: theory, applications and systems (BTAS’08), Arlington, VA, USA, pp 1–8. https://doi.org/10.1109/btas.2008.4699343

  • Singh YN, Gupta P (2009) Biometric method for human identification using electrocardiogram. In: Proceedings of the 3rd IAPR/IEEE international conference on biometrics, ICB 2009, Springer, Berlin, LNCS 5558, pp 1270–1279

  • Singh YN, Gupta P (2011) Correlation based classification of heartbeats for individual identification. J Soft Comput 15(3):449–460

    Google Scholar 

  • Singh YN, Singh SK, Gupta P (2012) Fusion of electrocardiogram with unobtrusive biometrics: an efficient individual authentication system. Pattern Recognit Lett 33(14):1932–1941

    Google Scholar 

  • Song Q, Du D, Yang L, Meng G, Mao X (2014) Drop fingerprint recognition method based on cluster analysis and BP neural network. Appl Mech Mater 543–547:2099–2102

    Google Scholar 

  • Sridharan K, Komarasamy G (2019) Sentiment classification using harmony random forest and harmony gradient boosting machine. Soft Comput. https://doi.org/10.1007/s00500-019-04370-z

    Article  Google Scholar 

  • Srihari SN, Srinivasan H (2008) Comparison of ROC and likelihood decision methods in automatic fingerprint verification. Int J Pattern Recognit Artif Intell 22(3):535–553

    Google Scholar 

  • Tang T (2012) Fingerprint recognition using wavelet domain features. In: Eighth international conference on natural computation (ICNC), Chongqing, China. https://doi.org/10.1109/icnc.2012.6234738

  • Tico M, Immonen E, Ramo P, Kuosmanen P, Saarinen J (2001) Fingerprint recognition using wavelet features. ISCAS 2001. In: The 2001 IEEE international symposium on circuits and systems (Cat. No.01CH37196), Sydney, NSW, Australia. https://doi.org/10.1109/iscas.2001.920996

  • Tun HM, Moe WK, Naing ZM (2015) Analysis of computer aided identification system for ECG characteristic points. Int J Biomed Sci Eng 3(4):49–61

    Google Scholar 

  • Um FO, Okoro CN (2012) Evaluating measures of indicators of diagnostic test performance: fundamental meanings and formulars. J Biom Biostat 3(1):1–10

    Google Scholar 

  • Wan Y, Yao J (2008) A neural network to identify human subjects with electrocardiogram signals. In: Proceedings of the world congress on engineering and computer science, WCECS 2008, San Francisco, USA

  • Wang Y, Agrafioti F, Hatzinakos D, Plataniotis KN (2008) Analysis of human electrocardiogram for biometric recognition. EURASIP J Adv Signal Process 2008:148658

    MATH  Google Scholar 

  • Wang J, Li J, Cao L (2010) An Improved fast thinning algorithm for fingerprint image and its application. J Comput Inf Syst 6(4):1027–1035

    Google Scholar 

  • Winston JJ, Hemanth DJ (2018) A Comprehensive review on Iris image-based biometric system. Soft Comput 23(19):9361–9384. https://doi.org/10.1007/s00500-018-3497-y

    Article  Google Scholar 

  • Woods K, Kegelmeyer WP, Bowyer K (1997) Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Anal Mach Intell 19(4):405–410

    Google Scholar 

  • Xu L, Krzyzak A, Suen CY (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybern 22(3):418–435

    Google Scholar 

  • Zhang Z, Hong WC (2019) Electric load forecasting by complete ensemble empirical mode decomposition adaptive noise and support vector regression with quantum-based dragonfly algorithm. Nonlinear Dyn 98:1107–1136

    Google Scholar 

  • Zhang Q, Yin Y, Zhan D, Peng J (2014) A novel serial multimodal biometrics framework based on semisupervised learning techniques. IEEE Trans Inf Forensics Secur 9(10):1681–1694

    Google Scholar 

  • Zokaee S, Faez K (2012) Human identification based on electrocardiogram and palmprint. Int J Electr Comput Eng (IJECE) 2(2):261–266

    Google Scholar 

Download references

Acknowledgements

This research was funded by the Deanship of Scientific Research at Princess Nourah bint Abdulrahman University through the Fast-track Research Funding Program.

Author information

Authors and Affiliations

  1. Computer Science Department, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia

    Sahar A. El_Rahman

  2. Electrical Engineering Department, Faculty of Engineering-Shoubra, Benha University, Cairo, Egypt

    Sahar A. El_Rahman

Authors
  1. Sahar A. El_Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sahar A. El_Rahman.

Ethics declarations

Conflict of interest

None to declare.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El_Rahman, S.A. Multimodal biometric systems based on different fusion levels of ECG and fingerprint using different classifiers. Soft Comput 24, 12599–12632 (2020). https://doi.org/10.1007/s00500-020-04700-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-04700-6

Keywords

Search

Navigation