3D face recognition based on pose and expression invariant alignment☆
Graphical abstract
Introduction
Biometric data tends to classify human beings through their distinctive physiological and behavioral characteristics. Among well-known biometric modalities like iris, voice, fingerprint, palm print and gait, human face is considered an effective biometric for a wide range of vital applications in numerous and diverse domains. Biometric data acquisition process has always been thought as an affront to privacy of a subject and requires subject’s cordial cooperation whereas facial biometric data is noninvasive and socially well accepted. Although face recognition using 2D intensity images has been studied intensively in the last decades and the majority of implemented face recognition systems are based on 2D images [1], it is still a very challenging task to recognize people using 2D images under diverse circumstances of pose, expression and illumination variations. With the emergence of reliable and inexpensive 3D scanners and the use of 3D facial structure information, the researchers generally conclude that some of the nuisance factors related to 2D images can be overcome by using 3D scans due to the reason that 3D data can more easily be pose corrected and it is not affected by illumination changes [2]. However, to handle facial expression variations, we still require evolution of algorithms, irrespective of using 2D or 3D data, because expressive faces complicate the face recognition by creating higher intra-class variance than inter-class variance. The main algorithms which have been evolved for 2D face recognition are based on holistic and local features of a facial image. In holistic techniques, face recognition is performed using the entire face. Three leading algorithms in this category include PCA, Linear Discriminant Analysis (LDA) and Independent Component Analysis (ICA) [3]. The major local feature based approaches of 2D face recognition are Local Binary Pattern (LBP) [4], Scale Invariant Feature Transform (SIFT) [3] and Gabor Wavelets [5]. The appearance based paradigm has also been explored to find strength of 3D face recognition and the most popular, appearance based 3D methods are PCA, LDA and ICA [1]. Local feature based 3D matching exploits properties of local descriptive points, curves and regions. The study [1], [6] reported point based methods that exploited meshSIFT algorithm to detect facial key points. Another study [7] explored the curvelet transform for detecting salient points on the face scans to build multi-scale local surfaces. The curve based method [8] proposed a Riemannian framework for analyzing facial shapes from radial curves emanating from the nose tip whereas the study [9] is a representative work of region based methods suitable for occlusion handling. The most crucial phase of any 3D face recognition algorithm is registration of facial surfaces and the final accuracy of the results greatly depends on the quality of the alignment module. In this work, we aim to present (i) an efficient face registration method based on intrinsic coordinate system and (ii) a face recognition method based on ensemble classifier that incorporates rank based fusion.
The first contribution of our study is a novel 3D registration algorithm. The proposed algorithm does not register two face scans to each other using the conventional registration process, rather transforms each 3D scan to an intrinsic coordinate system. This reference coordinate system is based on the nose tip, vertical symmetry plane and horizontal nose plane of the face. A novel method is presented for finding vertical symmetry plane and horizontal nose plane. Angles are measured between intrinsic and world coordinate systems which are then used to align the face surface. A novel method with mathematical validation is presented for quantitative analysis of our proposed registration method which is the second contribution of this study. The third contribution is a region based two tier, ensemble classification method where face is divided into regions like other approaches [10], [11]. PCA based classifier using MahCos distance metric is implemented for classification of each region where PCA is used to extract compact feature vectors. PCA is a subspace method which provides dimensionality reduction while relying on a set of basis vectors which correspond to maximum variance direction of the image data [12]. It is observed that combining results of multiple overlapping regions using an ensemble classifier produces excellent recognition results. For performance evaluation of proposed face registration and recognition method, GavabDB 3D database [13] of 61 subjects and FRGC v2.0 3D database [14] comprising of 4007 images from 466 subjects with pose and expression variations are used which are the most commonly used databases for 3D face recognition.
This study is organized as follows. Section 2 presents an overview of related work. Proposed methodology for 3D face registration and recognition is presented in Section 3. Section 4 is dedicated for experiments and results and a description of the used 3D face databases. Conclusion and future work is presented in Section 5.
Section snippets
Related work
The related work comprises of two parts. In the first part, work related to 3D face registration is addressed. The second part focuses on 3D face recognition, i.e. the classification of 3D facial data.
Proposed methodology
The framework of our proposed methodology is shown in Fig. 1 which presents whole 3D face recognition system.
The face registration, preprocessing, feature extraction and classification stages are depicted in the same figure.
Experiments and results
In this section two sets of experiments are reported for each of GavabDB [13] and FRGC v2.0 [14] 3D databases where each set includes one registration experiment explaining implementation of our registration method and two face recognition experiments conducted to evaluate the results of proposed face recognition method based on fusion of classifiers. These databases are briefly described as under.
Conclusion and future work
We presented 3D face registration and recognition algorithm where the robust, accurate and fast face registration method registers the 3D point cloud to the intrinsic coordinate system. Our algorithm is capable of correcting the 3D poses of neutral as well as expressive faces. Several novelties were contributed in this work explicitly: (i) Method for finding vertical symmetry plane and horizontal nose plane. (ii) Idea of representing reduced nose tip distance from 3D scanner as a measure of
Acknowledgments
The authors are thankful to the organizers of GavabDB: A.B. Moreno and A. Sanchez as well as the organizers of FRGC: J. Phillips, K. Bowyer and P. Flynn for provision of the datasets for research purposes.
Naeem Iqbal Ratyal received B.Sc. degree in electrical engineering from UAJ&K, Pakistan, in 1998, and M.Sc. degree in electrical engineering from UET Taxila, Pakistan, in 2008. Currently, he is a Ph.D. candidate at Mohammad Ali Jinnah University. His research interests are 3D face recognition, image processing and computer vision.
References (27)
- et al.
A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition
Comput Vis Image Underst
(2006) - et al.
A comparative study of texture measures with classification based on feature distributions
Pattern Recogn
(1996) - et al.
MeshSIFT: local surface features for 3D face recognition
Comput Vis Image Underst
(2013) - et al.
Face recognition based on 3D ridge images obtained from range data
Pattern Recogn
(2009) - et al.
Three-dimensional face recognition under expression variation
EURASIP J Image Video Process
(2014) - et al.
3D face recognition based on multiple keypoint descriptors and sparse representation
PLoS ONE
(2014) - et al.
An efficient multimodal 2D–3D hybrid approach to automatic face recognition
IEEE Trans Pattern Anal Mach Intell
(2007) - et al.
Distortion invariant object recognition in the dynamic link architecture
IEEE Trans Comput
(1993) - et al.
3-D face recognition using curvelet local features
IEEE Signal Process Lett
(2014) - et al.
3D face recognition under expressions, occlusions, and pose variations
IEEE Trans Pattern Anal Mach Intell
(2013)
3-D face recognition under occlusion using masked projection
IEEE Trans Inf Forensics Secur
Fast and accurate 3D face recognition using registration to an intrinsic coordinate system and fusion of multiple region classifiers
Int J Comput Vision
Regional registration for expression resistant 3-D face recognition
IEEE Trans Inf Forensics Secur
Cited by (18)
Resemblance of biological particle swarm optimization and particle swarm optimization for CBFR by using NN
2018, Materials Today: ProceedingsEfficient 3D face recognition using local covariance descriptor and Riemannian kernel sparse coding
2017, Computers and Electrical EngineeringCitation Excerpt :The recent expression-invariant 3D face recognition methods can be divided into three general strategies: (1) Face regions. The most commonly method for this issue is to select the defined structural region, such as the eyes-forehead and nose region [2], the T-region [3] and fusion of multiple regions [4]. In fact, the expression variation is relatively stable if the local region is small enough.
A novel statistical and multiscale local binary feature for 2D and 3D face verification
2017, Computers and Electrical EngineeringCitation Excerpt :ii) the ability of shape information to cope with many limitations of the photometry information. 3D scans are naturally less sensitive to lighting conditions and the free-form 3D facial surface makes it possible to correct head pose and face deformation more efficiently [5]. In addition, facial cosmetics tend to have less impact on 3D face scans. (
3D face recognition: A comprehensive survey in 2022
2023, Computational Visual MediaBPSO based neural network approach for content-based face retrieval
2022, Multimedia Tools and ApplicationsImage Alignment in Pose Variations of Human Faces by Using Corner Detection Method and Its Application for PIFR System
2022, Wireless Personal Communications
Naeem Iqbal Ratyal received B.Sc. degree in electrical engineering from UAJ&K, Pakistan, in 1998, and M.Sc. degree in electrical engineering from UET Taxila, Pakistan, in 2008. Currently, he is a Ph.D. candidate at Mohammad Ali Jinnah University. His research interests are 3D face recognition, image processing and computer vision.
Imtiaz Ahmad Taj received Ph.D. degree from Hokkaido University, Japan, in 2001. He is presently serving as Professor at Mohammad Ali Jinnah University and supervising research group of Vision and Pattern Recognition Systems. He has authored and co-authored 50+ research publications of international repute. He has supervised four Ph.D. dissertations in the fields of computer vision, pattern recognition and biometrics.
Usama Ijaz Bajwa is currently serving as an Assistant Professor at Department of Computer Science, COMSATS Institute of Information Technology. He did his M.S. & Ph.D. from CASE Islamabad, Pakistan. After completing his M.S., he served as a visiting researcher at Medical Imaging Lab, University of South Wales, United Kingdom. He has authored/co-authored 24 international conference and journal research papers.
Muhammad Sajid received his B.Sc. and M.Sc. degrees in electrical engineering from UAJ&K, Pakistan, in 2002 and 2008, respectively. He is currently pursuing Ph.D. at Mohammad Ali Jinnah University. His main research interests lie in the fields of face recognition, image processing and biometrics.
- ☆
Reviews processed and recommended for publication to the Editor-in-Chief by Associate Editor Dr. E. Cabal-Yepez.