Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Biometric recognition involves verifying a claimed identity based on physiological (e.g. face, fingerprint, veins, etc.) and behavioural characteristics (keystroke, mouse dynamics, gait, etc.). Biometrics has become a part of everyone’s daily life. The reliability and the accuracy of biometrics based systems, especially in constrained conditions have shown the benefit for applications such as border control, access control, forensics, etc. In spite of broad deployment of biometric systems, the accuracy of such systems has remained challenging in unconstrained conditions. Further, the vulnerability of biometric systems being attacked at various levels (sensor, comparison etc.) and the privacy concerns regarding stored biometric templates create a psychological barrier in accepting biometrics in the general sphere of life.

However, access control systems have started to employ smartphone based biometric authentication using face [5], iris [9] and periocular [7] to authenticate the user. Such systems further gained mainstream attention as major smartphone vendors like Apple, Samsung, HTC, etc. provided fingerprint-based authentication that can be used to unlock the smartphone and convenient access control. However, as the ease of presentation attacks on such sensors is known [2], fingerprint recognition should be used in applications including banking and finance transactions only if the transaction volume is limited. For higher transaction volumes face and periocular recognition are a better choice.

As the number of applications using smartphone access control continues to increase, the need to protect the biometric templates when stored on the device has become a task of utmost importance to preserve the privacy of the smartphone user. Even though device manufacturers have claimed to protect the biometric data by keeping the biometric templates in the secured hardware separated from the rest of the device, one cannot rule out the possibility of indirect attacks to retrieve stored biometric templates - for instance fingerprint templates stored in the smartphone [14]. The recent work reported in [14] has exposed the security loop-hole in HTC One Max smartphone where the fingerprint template was accessed from smartphone. Similar attacks carried out on Samsung Galaxy S5 smartphone to avail the fingerprint template further exemplifies the lapse of hardware based security [14]. These incidents illustrate the demand for an efficient biometric template protection scheme to ensure the user’s privacy and to avoid misuse of the stolen biometric templates on smartphones.

Recently significant progress has been made in smartphone based biometrics. The majority of the work is devoted to exploring the embedded sensors and designing efficient feature extraction and comparison algorithms of low complexity that are suitable for smartphones [1, 3]. However to the best of our knowledge, there is no reported work addressing biometric template protection on smartphones. In general, biometric template protection, which is one of the widely explored areas in biometrics, has resulted in a significant number of algorithms [10]. A very promising template protection approach using Bloom filters was proposed recently [11] and studied for different biometric characteristics such as iris [11] and face [12]. These studies have shown that it is possible to obtain privacy preserving protection which is not degrading the verification performance. Motivated by the lapse of hardware based smartphone security and leveraging the secure nature of Bloom filters for biometric templates, we introduce this template protection scheme for a smartphone based multi-biometric system. Our intention is to create the right balance between the accuracy and privacy, both of which are essential blocks for the success of smartphone based secure access control applications employing biometrics.

Fig. 1.
figure 1

Block diagram of the proposed multi-biometric template protection scheme for smartphones

Full size image

Template protection schemes can only be considered secure if they adhere to the principles of Irreversibility and Unlinkability, which are requirements according to the ISO/IEC 24745 standardFootnote 1.

Irreversibility - It should not be possible to reconstruct from a protected biometric template the original template or a biometric sample.

Unlinkability - It should be infeasible to link a subject’s protected biometric template from one application to another.

The rest of the paper is organized in the following manner: Sect. 2 introduces the proposed approach for multi-biometric template protected system on smartphone. Section 3 provides the details on the experimental set-up, evaluation protocols and the obtained results. Section 4 provides the concluding remarks of the current work and identifies the possible future work.

2 Proposed Multi-biometric Approach

This section provides the details of the proposed approach to enable a multi-biometric template protection system on smartphone. The proposed system includes the extraction of statistical image features from the captured face and periocular images and binarization of the extracted feature vectors followed by the transformation to protected templates using Bloom filters. The choice of Bloom filters is motivated by the non-degraded biometric performance while securing the biometric template as reported in [4].

Figure 1 shows the block diagram of the proposed biometric template protection scheme. The proposed system consists of three principal steps: (1) Segmentation of face and periocular region (left and right) from the captured image. (2) Generalized feature extraction and binarization scheme to extract the discriminant binary features from all segmented image regions. (3) Template protection using Bloom filters.

2.1 Face and Periocular Segmentation

In this work, we have carried out the face and periocular segmentation based on the Haar cascade classifier [13]. We have implemented two different classifiers as proposed in [8] to segment the face and periocular region (left and right) in real-time. Figure 1 shows an example and the results for the segmented face and periocular regions. The segmented face images are resized to \(64\times 80\) pixels and the periocular image to a size of \(120\times 88\) pixels that will be used for feature extraction. The parameters used for the resize step are selected based on the testing database (see Sect. 3) by considering both accuracy and the need for low complexity computations on the smartphone.

2.2 Generalized Feature Extraction and Binarization

The selection of a suitable feature extraction technique plays an important role to achieve both biometric template protection and also the expected verification performance on the smartphone. The Bloom filter, as many other template protection approaches, requires binary feature vectors as input. The main challenge is to use a low computation based binary feature extraction scheme that can provide good verification accuracy on both face and periocular biometrics. In this work we explore the binarized statistical image features (BSIF) [6] that can provide binary features by performing convolution with a number of filters that are learned on natural images. These statistically independent filters are optimized by exploring natural image statistics via unsupervised learning using Independent Component Analysis (ICA). Depending upon the various patch sizes and the number of ICA basis functions, one can learn a number of BSIF filters with various sizes (adjusted to the patch size) and bits (number of basis selected from ICA). In this work, we propose a feature extraction scheme based on eight different filter sizes such as: \(3 \times 3\), \(5 \times 5\), \(7 \times 7\), \(9 \times 9\), \(11 \times 11\), \(13 \times 13\), \(15 \times 15\) and \(17 \times 17\) and each filter is of 8 bit in length. The choices of the BSIF filter size and length is based on empirical trials on a testing dataset yielding higher verification accuracy (see Sect. 3).

Given a face (or periocular) image captured using the smartphone, we first divide the entire image into 32 different blocks such that each block size is of \(8\times 20\) pixels in the case of the face image and \(15\times 22\) pixels in the case of a periocular image. Then, for each block, we extract the BSIF features by convolving eight different BSIF filters each of 8-bit length. Finally the response of each filter with 8-bit size is encoded to a histogram of dimension \(1 \times 256\). Since we have eight different filters, the feature size for each block is \(8 \times 256\). We repeat this process for each block and concatenate the result to a final BSIF feature vector set of dimension 65536. The extracted histogram features h is binarized as follows:

$$\begin{aligned} h(i, j) = {\left\{ \begin{array}{ll} 1 &{} \quad \text {if } h(i,j) > 0 \\ 0 &{} \quad \text {if } h(i,j) = 0 \\ \end{array}\right. } \end{aligned}$$
(1)

We finally reorganize the binarized features to have a matrix of dimension \(4 \times 8\) rows and \(8 \times 256\) columns that can be used to generate a protected template using Bloom filter [4]Footnote 2.

2.3 Biometric Template Protection: Bloom Filters

In this work we have employed the Bloom filter by considering its irreversibility property in generating protected biometric templates [4]. The Bloom filter parameters are defined as follows: nBits denotes the number of bits used to address a location in a Bloom filter. nWords denotes the number of inputs to each Bloom filter. The length of each Bloom filter is defined by \(2^{nBits}\). Based on recommendations in [4] we use \(nBits = 4\), which makes the length of each Bloom filter 16-bits. The number of inputs to one Bloom filter is restricted to \(nWords \le 2^{nBits}\). In our case we set \(nWords = 16\).

Input to each Bloom filter is derived in the following manner: As described in the previous section, each template consists of 32 blocks of \(8 \times 256\) binary elements, \(x_{ij} \in {0, 1}\). To address a location in a Bloom filter of length 16, we further divide each of the 32 blocks into \(4 \times 16\) sub-blocks where each column (containing 4 binary values) is used as a codeword denoting an index in a Bloom filter. The value in the location indicated by the index is then set to 1. This approach will generate 1024 Bloom filters of length \(2^{nBits}\), achieving a final template of 2 kB.

To compare two templates we obtain a dissimilarity score using the distance metric as shown in Eq. 2 where |b| is the hamming weight or number of bits set to one in a Bloom filter, and \(HD(b_i, b_j)\) is the Hamming Distance between two Bloom filters or number of disagreeing pairs of bits [4].

$$\begin{aligned} DS(b_i, b_j) = \frac{HD(b_i, b_j)}{|b_i| + |b_j|} \end{aligned}$$
(2)

The unprotected and protected templates for face and periocular (left and right) regions are processed independently to obtain three separate dissimilarity scores. The final decision result is obtained by taking the majority vote on the decisions from each of the three scores.

Table 1. Quantitative results of proposed scheme on testing database
Full size table

3 Experimental Setup and Results

In this section we present the results of the proposed biometric template protection scheme for face and periocular biometrics on a smartphone. First, we evaluate the proposed system on a newly collected multi-biometric database using the Samsung Galaxy S5 smartphone constituting of 94 subjects. Table 1 shows the results from evaluation on the testing database in terms of Genuine Match Rate (GMR) for a fixed value of False Match Rate (FMR) and Equal Error Rate (EER). We observe that there is some performance degradation after Bloom filter is applied. However, with fusion of the different biometric characteristics we obtain a GMR of \(91.61\%\) when FMR = \(0.01\%\), indicating robustness and applicability of the proposed template protection scheme on smartphones for multi-biometrics. Then, a real-time evaluation, described in the next section, is carried out by providing the template protected biometric system for access control to 22 subjects on a Samsung Galaxy S5.

3.1 Real-Time Evaluation of the Proposed System

In this experiment the smartphone application was distributed on a Samsung S5 phone to 22 unique subjects along with the instruction to use the proposed system in order gain access to a secure application using multi-biometric (face and periocular) recognition. The threshold for genuine accept rate was set on the basis of threshold obtained on testing set at an FAR of \(0.1\%\). The subjects were instructed to enrol into the system with 10 images and they were further instructed to test the application independently in various lighting conditions for a number of times.

Fig. 2.
figure 2

Genuine and impostor attempts in real-time application of proposed system. Green color indicated number of successful attempts and red color indicates the number of failed attempts to login to the system. (Color figure online)

Full size image

On average 10 genuine attempts were recorded for the entire subset of users. The users were also asked to attempt to login as impostors to some other subjects than themselves to gauge the robustness of the system against zero-effort impostor attempts. These subjects repeatedly tried to gain impostor access for a period of 4 days. Experiments were carefully designed and the users were asked to attack target subjects that had the same ethnical background, for instance, an American subject was asked to pose as another compatriot to account for the robust impostor attacks.

Figure 2a shows the number of successful attempts when the system was not operating with protected templates and Fig. 2b shows the impostor attempts. The section indicated in the green color signifies the successful genuine attempts while the sections in red color indicate the rejection of the attempts for both genuine subjects and impostors. It can be observed from Fig. 2c and 2d that the system with the proposed template protection scheme works under operational conditions as good as the system without template protection. Very equivalent performance in both cases indicates the robust nature of the proposed template protection scheme while not compromising the verification accuracy significantly.

Based on the extensive analysis conducted from the above experiments, it can be observed that the proposed template protection system for multi-biometric based smartphone application is promising with an acceptable degradation of verification accuracy.

4 Conclusion

Preserving privacy of the smartphone user’s biometric templates is of utmost importance as the biometric characteristics cannot be renewed in case of compromise arising out of attacks on the biometric references. Wide spread use of smartphones for biometrics based authentication further justifies the need for protecting the templates on smartphones. In this work we have proposed a novel system using the strengths of Bloom filter and generalizability of BSIF on both face and periocular characteristics to devise a template protected biometric system for smartphones. Extensive experiments conducted on the database of 94 subjects indicate high verification accuracy even with protected templates. The GMR of \(91.61\%\) justifies the applicability of the proposed system for a real-life authentication scenario. Further, evaluation using disjoint set of subjects to test the reliability of system indicates the robust nature of proposed system against zero impostor attempts even with same ethnical origin.