Parallel score fusion of ECG and fingerprint for human authentication based on convolution neural network

Abstract

Biometrics have been extensively used in the past decades in various security systems and have been deployed around the world. However, all unimodal biometrics have their own limitations and disadvantages (e.g., fingerprint suffers from spoof attacks). Most of these limitations can be addressed by designing a multimodal biometric system, which deploys over one biometric modality to improve the performance and make the system robust to spoof attacks. In this paper, we proposed a secure multimodal biometric system by fusing electrocardiogram (ECG) and fingerprint based on convolution neural network (CNN). To the best of our knowledge, this is the first study to fuse ECG and fingerprint using CNN for human authentication. The feature extraction for individual modalities are performed using CNN and then biometric templates are generated from these features. After that, we have applied one of the cancelable biometric techniques to protect these templates. In the authentication stage, we proposed a Q-Gaussian multi support vector machine (QG-MSVM) as a classifier to improve the authentication performance. Dataset augmentation is successfully used to increase the authentication performance of the proposed system. Our system is tested on two databases, the PTB database from PhysioNet bank for ECG and LivDet2015 database for the fingerprint. Experimental results show that the proposed multimodal system is efficient, robust and reliable than existing multimodal authentication algorithms. According to the advantages of the proposed system, it can be deployed in real applications.

Introduction

Traditional authentication tactics like passwords and smart cards are insufficient for personal authentication because they can be shared, forgotten, copied, manipulated or forged. Unlike the traditional methods, the biometric system, which is the science of recognizing an individual based on his/her physiological or behavioral traits is beginning to gain acceptance as a legitimate method for determining an individual's identity. Nowadays, biometrics are no longer confined to criminal law enforcement. In addition, more businesses use biometrics to regulate access to buildings and information. However, most of the unimodal biometrics suffer from limitations such as noisy data, non-universality and spoof attacks (Jain and Ross, 2004), which makes it not be able to achieve the performance requirements of real-world applications. To overcome these drawbacks of unimodal systems, we proposed a novel multi-biometric system based on ECG and fingerprint. Our system overcomes the limitations of both single systems, improve the performance of the overall system and enhance the security. One of the main reasons for employing fingerprint with ECG is that the ECG signals can easily be acquired from fingers, which make the system very convenient and efficient to the acquisition of the system information, unlike other multimodal biometric systems that would be very inconvenient (e.g., a face, ear and fingerprint based multimodal biometric system). Moreover, ECG characteristics are suitable for human authentication as it provides the assurance of the aliveness of the person unlike other biometrics. Besides, fingerprints are one of the biometric authentication methods that have been widely used in many applications, and recently it gives acceptable accuracy for liveness detection (Kim et al., 2016, Nogueira et al., 2017).

Several multimodal biometric systems based on conventional traits such as; face and irises etc. have been developed during past decades (Rathgeb and Busch, 2014, Bailey et al., 2014, Saevanee et al., 2015); there are only a few works about a multimodal biometric system that includes ECG with fingerprint (Komeili et al., 2018, Singh et al., 2012, Zhao et al., 2012, Manjunathswamy et al., 2014). However, in these studies (Komeili et al., 2018, Singh et al., 2012, Zhao et al., 2012, Manjunathswamy et al., 2014), the authors concentrated on the conventional machine learning approaches, which often suffer from overfitting and show lower performance when validated on a separate dataset. In this study, we did not follow the conventional process by building the proposed multimodal biometric system based on convolutional neural network (CNN).

Recently, CNN has been employed in multimodal biometric systems (Al-Waisy et al., 2018, Talreja et al., 2017, Al-Waisy et al., 2017). However, none of the previous multimodal systems based on CNN working on ECG with the fingerprint for authentication. In this paper, we proposed a multimodal system using CNN based on a score level fusion of ECG and fingerprint for human authentication.

In this paper, we proposed a novel multimodal system using CNN based on score level fusion of ECG and fingerprint for human authentication. The feature extraction for individual modalities are performed using CNN and then biometric templates are generated from these features. After that, we have applied the matrix operation technique, which is one of the cancelable biometric techniques (Mukhaiyar et al., 2014) to protect these templates and increase the security of the proposed system. In the authentication stage, we proposed Q-Gaussian multi-class support vector machine (QG-MSVM) (Hammad and Wang, 2017) as a classifier for authentication to improve the performance. Finally, we used score level fusion to make the final decision. We have employed two databases: the Physikalisch-Technische Bundesanstalt (PTB) database (Goldberger et al., 2000) for ECG and LivDet2015 database (Mura et al., 2015) for the fingerprint to evaluate the performance of the proposed multimodal biometric system.

The main contributions of this paper can be summarized as follows:

• We have proposed an efficient feature extraction, which takes advantage of CNN to extract the deep features from the ECG without any preprocessing and segmentation steps. Also, we have employed the pre-trained deep CNN models for fingerprint authentication, where we used VGG-Net as the feature extractor, which achieves superior results compared with the previous hand-designed works.

• We are the first to apply the matrix operation technique to protect the deep features from ECG and to enhance the accuracy of authentication. Also, we applied it to protect the deep features of the fingerprint.

• We are the first to fuse ECG with fingerprint using CNN for human authentication. Where all previous multimodal biometric systems that used CNN are worked on other biometrics.

• We proposed QG-MSVM as a classifier to improve the authentication accuracy of the proposed system.

• Data augmentation is successfully used to increase the robustness of the proposed system against small variations.

The rest of this paper is organized as follows: Section 2 briefly reviews the previous works on ECG authentication, Fingerprint authentication as well as the fusion of ECG and fingerprint. Section 3 explained the details of the proposed approach including ECG authentication, fingerprint authentication, score level fusion of ECG and fingerprint and feature template updating. In Section 4, we describe the two datasets used in the experiments and evaluate the proposed system on these datasets. The experimental results are analyzed in Section 5. Finally, Section 6 concludes the paper.