Face spoofing detection with local binary pattern network☆
Introduction
Face recognition system has become prevalent in a broad range of applications [1], [2]. However, face spoofing attacks are always considered as serious threats to these systems [3], [4]. Specially, improvements on information acquisition make spoofing algorithms much easier to be designed, leading to the situation even more severe. In this sense, it is inevitable to develop effective spoofing detection (also called face anti-spoofing) methods and integrate them with biometric systems to prevent such frauds.
A face spoofing attack occurs when a person tries to masquerade as someone else by falsifying face and thereby attempting to gain illegitimate access and advantages. Based on different fake faces, four types of attacks can be considered: (i) printed face photos, (ii) displayed face images, (iii) replayed videos and (iv) 3D masks. In printed face photo attacks, the attacker prints the face photo on paper and puts it in front of the camera. In both displayed image and replayed video attacks, a digital screen is used to show the face images or replayed videos. For 3D mask scenario, the attacker uses a 3D mask of authorized person to enter the system. Among above mentioned face spoofing attack scenarios, the printed face photo attacks, displayed face image attacks, and 3D mask attacks cannot exhibit facial aliveness signals, such as eye blinking, lip movements and facial expression changes. Therefore, detecting replayed video attacks and distinguishing them from real faces are more challenging. However, it is also possible to include some aliveness information even in the printed face photos and 3D masks. For instance, the attacker may cut the eyes area in the mask in order to imitate the eye blinking pattern of the authorized subject. Fig. 1 shows an example for different face spoofing attacks.
Due to the printing defects and noises in cameras’ systems, fake faces may have some disadvantages such as lower image quality and lack of high frequency information, which can be presented by texture property [2], [7]. Therefore, facial texture analysis has attracted the attention of research community to tackle the problem of face spoofing attacks [8], [9], [10], [11], [12], [13], [14]. Especially with the success of deep learning in computer vision and multimedia analysis tasks [15], [16], [17], [18], [19], deep texture analysis based detection algorithms have also been applied in the face anti-spoofing problems [20], [21]. However, the fake face data is limited, which makes it difficult to train a deep network and severely constrains the detection performances. Compared with deep learning, the hand-crafted Local Binary Pattern (LBP) [22] features and its variations have stronger characterization ability in texture analysis [23], [24], [25], [26] and have been applied in face spoofing detection [5], [13], [14]. In fact, the essence of LBP extraction is to convolve the image with a set of fixed convolutional filters. As illustrated in Fig. 2, it shows how to use the convolutional filters to realize the process of LBP extraction. Considering the similarities between convolutional network and LBP extraction, we propose a new network structure for face spoofing detection.
Usually, training a deep network with fully learnable parameters and limited data is computationally expensive and prone to over-fitting. So, some variants of convolutional neural networks, such as using binary and fixed weights, have been proposed to tackle this problem [27], [28]. However, the parameters in fully connected layers are ignored. More importantly, there are more parameters in fully connected layers compared to convolutional layers. For instance, in VGG-face model [29], the parameter magnitude of fully connected layer is up to tens of millions (), whereas there are only 1728 parameters in the first convolutional layer. Apart from adopting binary and fixed weights, Hubara et al. [30] introduced some binary activate functions in their network. Based on binary weights and binary activate functions, their network can drastically reduce the memory consumption. However, unlike them, the purpose of our network is to reduce the number of training parameters.
The architecture of our proposed network is illustrated in Fig. 3. In our network, we save the parameters in convolutional layers and fully connected layers by introducing completely LBP layers. More specifically, in LBP layers, we use fixed sparse binary filters to save the parameters in convolutional layers and utilize a novel statistical histogram function to save the parameters in fully connected layers. We train and test our proposed method on two public available databases: Replay-Attack and CASIA-FA [7]. The experimental results demonstrate the effectiveness of our proposed method in face spoofing detection outperforming the state-of-the-art approaches.
The remainder of the paper is organized as follows: Section 2 reviews the existing state-of-the-art methods of face spoofing detection. Then our proposed learnable LBP network is introduced in Section 3. Section 4 provides the details of experimental setup. After that, we show and analyze the detection results in Section 5. Finally, in Section 6, we conclude this paper and discuss the directions for future research.
Section snippets
Related work
Since the early 2000s, many face spoofing detection methods have been proposed [5], [6], [7], [13], [31], [32], [33], [34], [35], [36], [37]. Based on different clues, we categorize these methods into four categories: (i) texture analysis [5], [13], [31], (ii) motion analysis [32], [33], [38], (iii) image quality analysis [7], [34], [35], and (iv) hardware based methods [6], [36], [37].
Overall architecture
In order to tackle the problem of insufficient training data and excessive network parameters, we propose a novel end-to-end learnable LBP network for face spoofing detection. By integrating fixed sparse binary filters and derivable statistical histogram functions, our proposed network has three distinctive advantages: (i) drastically reducing the network parameters in convolutional and fully connected layers; (ii) effectively training the parameters directly with limited data; (iii) completely
Experiment data
We validate our proposed method on two public available face anti-spoofing databases: Replay-Attack [5] and CASIA Face Anti-spoofing [7]. Table 2 gives a comparison about these two databases, and a description of each database is given below.
Tested on replay-attack and CASIA-FA databases
To evaluate the effectiveness of our proposed LBP network, we perform experiments on Replay-Attack and CASIA-FA databases. More specifically, we test on three kinds of attacks: printed photos, displayed images and replayed videos. For CASIA-FA, there are two kinds of print fake faces: warped photos and cut photos. Caused by the impact of eye blinking, the experiments of CASIA-FA are conducted on warped photo attacks and cut photo attacks, respectively.
Table 3 shows the detection results of
Conclusion
We proposed to approach the problem of face spoofing detection by a novel LBP network. The proposed network combines the hand-crafted features with deep learning and can reduce the network parameters by obtaining the statistical histograms. Extensive experiments on Replay-Attack and CASIA-FA databases showed interesting results. More importantly, unlike most of the state-of-the-art methods, our proposed method can achieve stable performances on the both databases. However, our proposed network
Conflict of interest
There is no conflict of interest.
Acknowledgment
This work is partly supported by the National Aerospace Science and Technology Foundation and the National Nature Science Foundation of China (No. 61702419).
References (66)
- et al.
Deep convolutional hashing using pairwise multi-label supervision for large-scale visual search
Signal Processing Image Commun.
(2017) - et al.
Integration of image quality and motion cues for face anti-spoofing
J. Visual Commun. Image Representation
(2016) - Z. Akhtar, G. Fumera, G.L. Marcialis, F. Roli, Evaluation of serial and parallel multibiometric systems under spoofing...
- et al.
Face spoof detection with image distortion analysis
IEEE Trans. Inform. Forensics Security
(2015) - Y. Li, K. Xu, Q. Yan, Y. Li, R.H. Deng, Understanding osn-based facial disclosure against face authentication systems,...
- L. Omar, I. Ivrissimtzis, Evaluating the resilience of face recognition systems against malicious attacks, in: Seventh...
- I. Chingovska, A. Anjos, S. Marcel, On the effectiveness of local binary patterns in face anti-spoofing, in: Biometrics...
- N. Erdogmus, S. Marcel, Spoofing in 2d face recognition with 3d masks and anti-spoofing with kinect, in: IEEE Sixth...
- Z. Zhang, J. Yan, S. Liu, Z. Lei, D. Yi, S.Z. Li, A face antispoofing database with diverse attacks, in: International...
- N. Kose, J.L. Dugelay, Classification of captured and recaptured images to detect photograph spoofing, in:...
Face liveness detection from a single image via diffusion speed model
IEEE Trans. Image Processing
Deep hashing for scalable image search
IEEE Trans. Image Processing
Scarce face recognition via two-layer collaborative representation
IET Biometrics
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
IEEE Trans. Pattern Anal. Mach. Intell.
Face recognition using local binary patterns (lbp)
Global J. Comput. Sci. Technol.
Learning compact binary face descriptor for face recognition
IEEE Trans. Pattern Anal. Mach. Intell.
Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition
IEEE Trans. Pattern Anal. Mach. Intell.
Binaryconnect: Training deep neural networks with binary weights during propagations
Adv. Neural Inf. Processing Syst.
Binarized neural networks
Adv. Neural Inf. Processing Syst.
Cited by (61)
Multiscale residual gradient attention for face anti-spoofing
2023, Journal of Visual Communication and Image RepresentationHistNet: Histogram-based convolutional neural network with Chi-squared deep metric learning for facial expression recognition
2022, Information SciencesCitation Excerpt :As mentioned in the last paragraph of Section 2 (related work), some efforts have been performed to use histogram calculation in CNNs; however, these studies used conventional methods for the classification stage. For example, Sedighi and Fridrich [34] employed a simple two-layer neural network for classification; similarly, in [35], a SVM classifier was used for classification of the histogram of LBP codes extracted by a CNN. In this section, a novel approach has been proposed to use for the classification of the calculated histograms from the facial expression feature maps.
Face spoofing detection ensemble via multistage optimisation and pruning
2022, Pattern Recognition LettersFacial Expression Recognition with High Response-Based Local Directional Pattern (HR-LDP) Network
2024, Computers, Materials and Continua
- ☆
This paper has been recommended for acceptance by Jiwen Lu.