Robust face recognition under partial occlusion based on support vector machine with local Gaussian summation kernel
Introduction
Face recognition has many potential applications, for example, to security systems, man–machine interfaces, and searches of video databases or the WWW. Therefore, many researchers are actively working in this field and many face recognition methods have been proposed [1], [2]. At present, human faces can be recognized with high accuracy in a restricted environment [1], [2], [3], [4]. However, face recognition in practical environments is still problematic, with obstacles such as occlusion, illumination changes and pose changes to be overcome. A robust recognition method for use in practical environments is therefore desired. Pose changes of faces induce large variation in feature space [5], [6], [7], making pose-independent recognition difficult. However, some studies have realized pose-independent face recognition by combining certain pose-dependent classifiers [7], [8], [9], [10]. It is important to make pose dependent classifier robust, especially to occlusion, because human faces are sometimes occluded by the wearing of sunglasses, scarves and other items in practical environments. Shadows on faces due to changes in illumination are also considered as a kind of occlusion. Therefore, robustness to occlusion would further improve face recognition accuracy in practical environments.
In recent years, the effectiveness of Support Vector Machines (SVMs) [11], [12] has been reported [3], [13], [14]. However, because conventional methods apply one kernel to global features extracted from one image [3], [4], [14] and global features are influenced easily by noise or occlusion, conventional methods are not robust to occlusion. Effectiveness of recognition methods based on local features has also been reported in recent years [9], [10], [15], [16], [17]. Since partial occlusion affects only specific local features. Local features based methods are expected to be robust to partial occlusion if local features are integrated well. For example, Martinez realized robust recognition under partial occlusion by integrating the local similarities [18]. Thus, in order to give SVM robustness under partial occlusion, it is necessary to treat local features in SVM. In this paper, SVM with local kernels is proposed [19]. In this approach, local kernels are arranged at all local regions of a recognition target and are used in SVM to realize robust face recognition under partial occlusion.
Fig. 1(a–c) show a face image, the conventional global kernel and local kernels, respectively. The circle represents one kernel. SVM requires one kernel value when two images are given. However, many outputs of local kernels are obtained in the case of Fig. 1(c) and the outputs of local kernels must therefore be integrated. Both the product and summation of local kernels are considered integration methods that satisfy Mercer’s theorem [20], [12]. We will refer to these methods as the local product kernel and local summation kernel, respectively. It is considered that the local summation kernel is better than the local product kernel. The reason is as follows. In the product case, if only one local kernel gives a value of nearly zero, then the product kernel value becomes nearly zero. This means that the product kernel is influenced easily by noise or occlusion. On the other hand, the summation kernel value is not influenced when some local kernels give a value of nearly zero, meaning that the summation of local kernels is robust to occlusion. Research on classifier combination strategies has found that integration by summation gives the best result [21]. Thus, summation is used for integrating local kernels in the proposed method.
To investigate robustness to partial occlusion, we use face images to which a white or black square is added randomly. Effectiveness and robustness of the proposed method are shown by comparison with global kernel based SVM. The recognition rate of the proposed method is shown to be high under large occlusion, whereas the recognition rate of the SVM with the global Gaussian kernel decreases drastically. In addition, we investigate the robustness to practical occlusion in the real world by using the AR face database [22]. Although only face images with non-occlusion are used for training, faces wearing sunglasses or a scarf are classified with high accuracy. In the experiments, the proposed method is also compared with another robust recognition method, the weighted voting method using local eigenspaces, under partial occlusion [18]. Effectiveness of the proposed method is demonstrated by the comparison.
This paper is organized as follows. First, related works are described in Section 2. The contributions of the proposed method are clarified. In Section 3, a face recognition method based on SVM with local Gaussian summation kernel is explained. Section 4 presents the experimental results using artificially occluded face images and then demonstrates robustness to practical occlusion such as the wearing of sunglasses or a scarf. Conclusions and future work are described in Section 5.
Section snippets
Related works
A face recognition method based on multiple local SVMs was proposed in recent years [13]. Although the method pays attention to local features, SVM is applied to all features extracted from a local region, that is, the global kernel based SVM is applied to the local region. Thus it uses all features of local region, so is not robust to occlusion. The method proposed here is different in that it is based on local kernels.
Recognition methods based on SVM with local kernel have also been proposed
A robust recognition method under partial occlusion
This section explains the face recognition method based on SVM with a local Gaussian summation kernel. The proposed method is based on local kernels. In order to use local kernels effectively, we want to use local appearance features. For this purpose, we use Gabor features which give good performance in face recognition [29], [30], [26]. The flowchart of the proposed method is shown in Fig. 2. First, Gabor features are extracted from a recognition target. Second, local kernels are arranged at
Experiments
This section presents the experimental results. In Section 4.1, the effectiveness and robustness of the proposed method are evaluated using artificially occluded face images. The accuracy of global kernel based SVM is also evaluated for comparison. In addition, the proposed method is compared with the weighted voting method using local eigenspaces which is a robust recognition method under partial occlusion [18]. In Section 4.2, face images wearing sunglasses and a scarf are used to evaluate
Conclusions
This paper proposes SVM with local Gaussian summation kernel as a robust method of face recognition. Robustness to occlusion is obtained easily by summation of local Gaussian kernels. In addition, the global optimal solution is consistently obtained because the proposed kernel satisfies Mercer’s theorem. The effectiveness of the local Gaussian summation kernel is shown by comparison with global kernel based SVMs. The proposed method gives high accuracy under large artificial occlusion, whereas
Acknowledgements
I thank Prof. Haruhisa Takahashi for providing the opportunity for this work to be undertaken.
This work is supported in part by the Grant-in-Aid for Scientific Research (No.15700150) from the Ministry of Education, Culture, Sports, Science and Technology of Japan.
References (41)
- et al.
Viewpoint independent face recognition by competition of viewpoint dependent classifiers
Neurocomputing
(2003) - et al.
The ‘independent components’ of natural scenes are edge filters
Vision Research
(1997) - et al.
Human and machine recognition of faces: a survey
Proceedings of the IEEE
(1995) - et al.
Face recognition: a literature survey
ACM Computing Surveys
(2003) - G. Guo, S. Li, K. Chan, Face recognition by support vector machines, in: Proceedings of the Fourth IEEE International...
- M.-H. Yang, Face recognition using kernel methods, in: Advances in Neural Information Processing Systems 14, 2002, pp....
- P. Phillips, P. Grother, R. Micheals, D. Blackburn, E. Tabassi, J. Bone, Frvt2002: Evaluation report, Tech. report,...
- et al.
Visual learning and recognition of 3d objects from appearance
International Journal of Computer Vision
(1995) - et al.
A fast and accurate face detector based on neural networks
IEEE Trans Pattern Analysis and Machine Intelligence
(2001) - A. Pentland, B. Moghaddam, T. Starner, View-based and modular eigenspaces for face recognition, in: Proceedings of the...
Statistical Learning Theory
An Introduction to Support Vector Machines
Support vector machines for 3d object recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence
Face recognition: features versus template
IEEE Transactions on Pattern Analysis and Machine Intelligence
Detectability, uniqueness, and reliability of eigen windows for stable verification of partially occluded objects
IEEE Transactions on Pattern Analysis and Machine Intelligence
A view-invariant face detection method based on local pca cells
Journal of Advanced Computational Intelligence and Intelligent Informatics
Recognizing imprecisely localized, partially occluded, and expression variant faces from a single sample per class
IEEE Transactions on Pattern Analysis and Machine Intelligence
Cited by (76)
Pixel Based Supervised Classification of Hyperspectral Face Images for Face Recognition
2018, Procedia Computer ScienceRobust face detection using local CNN and SVM based on kernel combination
2016, NeurocomputingVector projection for face recognition
2014, Computers and Electrical EngineeringCitation Excerpt :Thus the biased estimation is useful for the proposed VPC approach. Partial occlusion is one of the benchmark problems in face recognition, numerous previous works have shown that the performance of holistic methods are very poor to solve partial occlusion problem, but local methods can achieve good performance [21–25]. In the LVPC algorithm, the face images are divided into a number of image blocks, and then each image block is processed by the VPC algorithm, the final classification is to find the best match based on the information of all image blacks by using a decision strategy.
A New Design of Occlusion-Invariant Face Recognition Using Optimal Pattern Extraction and CNN with GRU-Based Architecture
2023, International Journal of Image and GraphicsA New design of occlusion Invariant Face Recognition Using optimal Pattern extraction and CNN with GRU-Based Architecture
2022, International Journal of Information Security and Privacy