Recognizing occluded faces by exploiting psychophysically inspired similarity maps
Graphical abstract
Highlights
► Psychophysical principles. ► Intuitive occlusion model. ► Visualization of abstract phenomena via Bayesian Networks. ► Improved results over state of the art.
Introduction
Enhancing computer vision systems with psychophysical mechanisms enable them to take intelligent decisions on a par with the human mind, especially when uncertainty elements such as occlusions are encountered. A key objective of machine vision researchers is to build automated face recognition systems that can compete and eventually surpass, human performance. Many successful face recognition and verification models (Raducanu and Vitria, 2008, Schwaninger et al., 2004, Smeraldi and Bigun, 2002, Turk and Pentland, 1991) have derived their fundamental ideas from principles of cognitive psychology and neuroscience. Substantial research is ongoing to make face recognition systems robust to typical operational environments where uncertainties such as occlusion, illumination and other variations are common.
In this contribution we propose a novel “Psychophysically Inspired SImilarity MApping (PISIMA)” model to attack this potential problem. We differ from other approaches in the sense that, we intend to identify some key cognitive psychology principles from the literature which are vital to face recognition and transform them to a computational model with the aid of an intuitive Bayesian framework. This contribution is an enhanced version of the preliminary work carried out by Ibrahim et al. (2011). Basically PISIMA intends to map similarities between two main object entities. The first entity is a set of sparse components of the probe face image. The second entity comprises of a bulk set of known face images that are stored in the face database which are known as gallery samples. Few faces are recalled for each of the sparse components as a consequence of this similarity mapping based reasoning. Further this reasoning reveals inherent conditional independence properties between the subsamples of a face and the recalled face images which are influenced by the subsamples. Such conditional independent notions could be scientifically represented by Bayesian Networks (BNs) as mentioned by Nilsson (1998). Formally a BN is a Directed Acyclic Graph (DAG) where the nodes represent variables and the arcs encode conditional independencies between the variables. BNs serve as fundamental tools in tackling uncertainty problems as they characterize intuitive notions of human reasoning. In other words, PISIMA employs BNs to establish, learn and exploit intrinsic similarity mappings that are inherent in the face domain.
We will briefly present the framework of the proposed PISIMA with the aid of the flow-chart shown in Fig. 1. Firstly, face images in the database have been enhanced with standard preprocessing techniques. We have adapted the techniques proposed by Bartlett and Movellan (2002) to normalize the face images. This image enhancement enables the face images to be independent of variations such as scaling, translation, rotation and so on. Then a feature space (low dimensional face space) is constructed from the gallery (training set) of face images available in the face database using Principal Component Analysis (PCA). PISIMA further learns conditional probability potentials which provide information about how well a face can be influenced given that a particular face component or a combination of face components has been observed. This learning process is done offline from the gallery face images, that is when computing resources are free.
A given probe face is enhanced using similar preprocessing techniques which were applied to the gallery samples and subject to horizontal segmentation. Then the PCA features of facial entities, acquired by combining probe face components over the gallery face images, are extracted and projected over the feature space using an inheritance mechanism which will be described in Section 3.2. Further probable subjects are shortlisted by means of similarity mapping based processing. For a given probe, a BN is generated whose child node variables represent the belief states of short-listed subjects and the parent nodes represent the belief states of corresponding components which influenced them. Finally faces are recognized using a face score formula which will be derived in Section 3.4.
Section snippets
Synthesis of relevant ideas apart from computer vision
Apart from the pixel domain we have investigated some key principles relevant to face recognition from other related fields. Vision is a subfield of cognitive science which involves psychological inferences in the higher nervous system (Frey and Jojic, 2005, Lin et al., 2009). Psychologists are of the view that psychologically feasible computational models exhibit clear and strong relationships between behavior and properties of the domains which they intend to represent (Fific, 2005). A very
Building the Bayesian Network from facial components
In this section we will represent the facial domain in terms of simple components and gradually build the Bayesian Network from those components.
Experimental results and discussions
We have implemented the PISIMA model using MATLAB on Intel Pentium IV Core 2 Duo 2.39 Ghz CPU with 2 GB of RAM. We have made use of the routines offered by Delac et al., 2006, Murphy, 2007 to build the PCA subspace and BN respectively. We have evaluated the performance of PISIMA model using a series of experiments on standard Face Datasets (FDs). We have used the AR FD (Martinez and Benavente, 1998) which consists of over 3200 color images of 126 subjects. Images feature frontal view faces with
Conclusion
We have discovered that faces exhibit interesting similarity mappings via Bayesian Networks. The proposed PISIMA model encapsulates key psychophysical principles fundamental to reasoning objects under uncertainty, by means of statistical machine learning techniques. The proposed framework intuitively exploit these intrinsic mappings to recognize faces when they are prone to major occlusions coupled with other variations. Further the graphical nature of PISIMA aids to visualize the abstract
Acknowledgments
The authors thank those who have contributed in developing the AR dataset and for making them publicly available. We thank Dr. Murphy and his team, University of British Columbia, for providing the Bayes Net Toolbox. This research is supported by the James Watt scholarship awarded by Heriot-Watt University, UK. We also thank Kolej University TATI, Malaysia and Universiti Sains Malayisa for their support. We thank the editors and anonymous reviewers for providing useful suggestions.
References (55)
- et al.
Recognizing faces using adaptively weighted Sub-Gabor array from a single sample image per enrolled subject
Image Vision Comput.
(2010) - et al.
Learning to learn: From smart machines to intelligent machines
Pattern Recognition Lett.
(2008) - et al.
Retinal vision applied to facial features detection and face authentication
Pattern Recognition Lett.
(2002) - et al.
Overview on Bayesian Networks applications for dependability, risk analysis and maintenance areas
Eng. Appl. Artif. Intel.
(2012) A new look at the statistical model identification
IEEE Trans. Automat. Control
(1974)- et al.
Face recognition by independent component analysis
IEEE Trans. Neural Networks
(2002) - et al.
Face, gender and emotion recognition using holons
Adv. Neural Inf. Process. Syst.
(1991) - et al.
A Bayesian approach to object detection using probabilistic appearance-based models
Pattern Anal. Appl.
(2004) - et al.
Biological bar codes in human faces
J. Vision
(2009) - et al.
Cue saliency in faces as assessed by the photofit technique
Perception
(1977)
Independent comparative study of pca, ica, and lda on the feret data set
Internat. J. Imaging Systems Technol.
Pattern Classification
Identification of familiar and unfamiliar faces from internal and external features: Some implications for theories of face recognition
Perception
Reaction time measures of feature saliency in schematic faces
Perception
A comparison of algorithms for inference and learning in probabilistic graphical models
IEEE Trans. Pattern Anal. Machine Intell.
Face inversion disproportionately impairs the perception of vertical but not horizontal relations between features
J. Exp. Psychol. Human Percept. Perform.
Digital Image Processing
Two and Three Dimensional Patterns of the Face
A component-based framework for face detection and identification
Internat. J. Comput. Vision
Psychophysically inspired Bayesian occlusion model to recognize occluded faces
Bayesian Networks and Decision Graphs
Effective representation using ica for face recognition robust to local distortion and partial occlusion
IEEE Trans. Pattern Anal. Machine Intell.
Reliable face recognition using adaptive and robust correlation filters
Computer Vision and Image Understanding
Robust face recognition using posterior union model based neural networks
IET Comput. Vision
Recognizing imprecisely localized, partially occluded and expression variant faces from a single sample per class
IEEE Trans. Pattern Anal. Machine Intell.
Cited by (16)
Fast and robust occluded face detection in ATM surveillance
2018, Pattern Recognition LettersCitation Excerpt :A more comprehensive survey of existing works is presented in Section 2. Among various existing approaches [1,5,11,30,37–40], detecting users’ facial images are widely used in practice, where the acquired facial images are crucial for verifying occlusion. Some recent methods are proposed related to face recognition and classification [8–10,26].
Optimized symmetric partial facegraphs for face recognition in adverse conditions
2018, Information SciencesCitation Excerpt :The AR Face database has been used to evaluate the OSPF performance under different operational conditions. The performance of the proposed OSPF has been compared with ten standard approaches including Line Edge Map (LEM) [12], Ensemble String Matching (ESM) [8], Adaptively Weighted Patch Pseudo Zernike Moment Array (AWPPZMA) [20], partitioned Sparse Representation-based Classification (p-SRC), [43],Adaptively Weighted Sub-Gabor Array(AWSGA) [19] , Psychophysically Inspired Similarity MApping (PISMA) [40] and Harmony Search Oriented EBGM (HSO-EBGM) [24], Adaptive Noise Dictionary_Sparse Regression based Classifier (AND_SRC) [9], Local Structure based Multi-Phase Collaborative Representation Classification (LS_MPCRC) [28] and SIFT based Kepenekci approach (SIFT) [27]. In this section, the proposed OSPF approach has been assessed under relatively controlled conditions using both the AR and the FRGC ver2.0 databases.
Recognizing faces prone to occlusions and common variations using optimal face subgraphs
2016, Applied Mathematics and ComputationCitation Excerpt :Face recognitions systems have an essential role in biometric-oriented video surveillance systems which have been progressively incorporated in operational environments where the problem of encountering occlusions cannot be avoided [38]. Several approaches have been proposed to overcome the shortcomings of face recognition challenges: expressions, illumination conditions and facial occlusions [16,28,30,38,41,42,44]. In general, occluded face recognition approaches are classified into two categories: holistic and component-based.
Partial Face Identification using Local Feature Extraction Algorithm on Different Classifiers
2023, 2023 International Conference on Computer, Electronics and Electrical Engineering and their Applications, IC2E3 2023Recognition of partially occluded faces using regularized ICA
2021, Inverse Problems in Science and Engineering