Recognizing occluded faces by exploiting psychophysically inspired similarity maps

Abstract

The presence of occlusions in facial images is inevitable in unconstrained scenarios. However recognizing occluded faces remains a partially solved problem in computer vision. In this contribution we propose a novel Bayesian technique inspired by psychophysical mechanisms relevant to face recognition to address the facial occlusion problem. For some individuals certain facial regions, e.g. features comprising of some of the upper face, might be more discriminative than the rest of the features in the face. For others, it might be the features over the mid face and some of the lower face that are important. The proposed approach in this paper, will allow for such a psychophysical analysis to be factored into the recognition process. We have discovered and modeled similarity mappings that exist in facial domains by means of Bayesian Networks. The model can efficiently learn and exploit these mappings from the facial domain and hence capable of tackling uncertainties caused by occlusions. The proposed technique shows improved recognition rates over state of the art techniques.

Introduction

Enhancing computer vision systems with psychophysical mechanisms enable them to take intelligent decisions on a par with the human mind, especially when uncertainty elements such as occlusions are encountered. A key objective of machine vision researchers is to build automated face recognition systems that can compete and eventually surpass, human performance. Many successful face recognition and verification models (Raducanu and Vitria, 2008, Schwaninger et al., 2004, Smeraldi and Bigun, 2002, Turk and Pentland, 1991) have derived their fundamental ideas from principles of cognitive psychology and neuroscience. Substantial research is ongoing to make face recognition systems robust to typical operational environments where uncertainties such as occlusion, illumination and other variations are common.

In this contribution we propose a novel “Psychophysically Inspired SImilarity MApping (PISIMA)” model to attack this potential problem. We differ from other approaches in the sense that, we intend to identify some key cognitive psychology principles from the literature which are vital to face recognition and transform them to a computational model with the aid of an intuitive Bayesian framework. This contribution is an enhanced version of the preliminary work carried out by Ibrahim et al. (2011). Basically PISIMA intends to map similarities between two main object entities. The first entity is a set of sparse components of the probe face image. The second entity comprises of a bulk set of known face images that are stored in the face database which are known as gallery samples. Few faces are recalled for each of the sparse components as a consequence of this similarity mapping based reasoning. Further this reasoning reveals inherent conditional independence properties between the subsamples of a face and the recalled face images which are influenced by the subsamples. Such conditional independent notions could be scientifically represented by Bayesian Networks (BNs) as mentioned by Nilsson (1998). Formally a BN is a Directed Acyclic Graph (DAG) where the nodes represent variables and the arcs encode conditional independencies between the variables. BNs serve as fundamental tools in tackling uncertainty problems as they characterize intuitive notions of human reasoning. In other words, PISIMA employs BNs to establish, learn and exploit intrinsic similarity mappings that are inherent in the face domain.

We will briefly present the framework of the proposed PISIMA with the aid of the flow-chart shown in Fig. 1. Firstly, face images in the database have been enhanced with standard preprocessing techniques. We have adapted the techniques proposed by Bartlett and Movellan (2002) to normalize the face images. This image enhancement enables the face images to be independent of variations such as scaling, translation, rotation and so on. Then a feature space (low dimensional face space) is constructed from the gallery (training set) of face images available in the face database using Principal Component Analysis (PCA). PISIMA further learns conditional probability potentials which provide information about how well a face can be influenced given that a particular face component or a combination of face components has been observed. This learning process is done offline from the gallery face images, that is when computing resources are free.

A given probe face is enhanced using similar preprocessing techniques which were applied to the gallery samples and subject to horizontal segmentation. Then the PCA features of facial entities, acquired by combining probe face components over the gallery face images, are extracted and projected over the feature space using an inheritance mechanism which will be described in Section 3.2. Further probable subjects are shortlisted by means of similarity mapping based processing. For a given probe, a BN is generated whose child node variables represent the belief states of short-listed subjects and the parent nodes represent the belief states of corresponding components which influenced them. Finally faces are recognized using a face score formula which will be derived in Section 3.4.