An eye detection method robust to eyeglasses for mobile iris recognition

Highlights

• An eye detection method that is robust to eyeglass interference is proposed for mobile iris recognition system.

• Multi-scale window mask consisting of 2 × 3 subblocks is used to generate eye candidates.

• Eye validation is applied to select the true eye image among the eye candidates.

• The proposed method performs more accurately and quickly than competing methods in mobile environment.

Abstract

Finding the accurate position of an eye is crucial for mobile iris recognition system in order to extract the iris region quickly and correctly. Unfortunately, this is very difficult to accomplish when a person is wearing eyeglasses because of the interference from the eyeglasses. This paper proposes an eye detection method that is robust to eyeglass interference in mobile environment. The proposed method comprises two stages: eye candidate generation and eye validation. In the eye candidate generation stage, multi-scale window masks consisting of 2 × 3 subblocks are used to generate all image blocks possibly containing an eye image. In the ensuing eye validation stage, two methods are employed to determine which blocks actually contain true eye images and locate their precise positions as well: the first method searches for the glint of an NIR illuminator on the pupil region. If this first method fails, the next method computes the intensity difference between the assumed pupil and its surrounding region using multi-scale 3 × 3 window masks. Experimental results show that the proposed method detects the eye position more accurately and quickly than competing methods in the presence of interference from eyeglass frames.

Introduction

Biometric technology provides a robust level of identity verification with convenience and reliability. As a result, fingerprint recognition and facial recognition are being extensively adopted on mobile devices for tasks such as mobile device unlocking. However, demands for more rigorous and reliable identity verification methods, necessitated by e-commerce and mobile payment systems, have led to iris recognition being considered as the next biometric technology for user verification on mobile devices (Ashbourn, 2014).

Most iris recognition systems adopt near infrared (NIR) cameras with high resolution and powerful NIR illuminators (Ghinea, Djeraba, Gulliver, & Pernice Coyne, 2007). Iris images are captured with NIR cameras because iris images with sharper structural patterns can be obtained even from dark-colored irises with NIR cameras than with visible light cameras. After input images are obtained, eye detection is processed to locate the position of the iris. Different from eye detection in the face recognition, the iris with the size that is appropriate for iris recognition must be detected because the detected iris region should be sufficiently large that the pattern of the iris is clear enough. According to Int. Std. ISO/IEC 19,794-6 (Commission & Standards, 2011), an iris image which the iris diameter is larger than 200 pixels is considered as good quality and which the iris diameter is between 150 and 200 pixels is considered as adequate quality.

In mobile iris recognition, because of the limited resolution of the NIR camera and the limited power of the NIR illuminators, use of a single eye image is preferred to obtain a good quality iris image. If both eyes are used, the view angle of the camera could not cover both eyes without severely distorting the iris image. In addition, for user convenience, the first input image containing a good quality iris image is chosen automatically from among the continuous stream of input images without asking the user to make a selection. Thus, each input image has to be evaluated sufficiently quickly not to miss the good iris image for iris recognition as all the input images except the selected one are discarded in a real-time manner. Because of the low CPU facility within a mobile phone, an eye detection algorithm should have low computational cost in order to detect a good iris image in real time. Moreover, the eye position has to be determined precisely as it will subsequently affect iris boundary detection in the iris recognition system. Therefore, fast and accurate eye detection is crucial in mobile iris recognition system.

Throughout the past few decades, eye detection or finding the position of an eye has been employed in various applications including iris recognition, gaze estimation, face modeling, and intelligent user interface, because information about the eye position plays important roles in most face analysis tasks (Ji et al., 2005, Wildes, 1997). Numerous eye detection techniques have been developed to achieve highly accurate and robust performance. In general, eye detection methods can be divided into four categories according to geometric and photometric properties (Hansen and Ji, 2010, Song et al., 2013): 1) shape-based methods; 2) feature-based methods; 3) appearance-based methods; and 4) hybrid modeling methods.

Shape-based methods use the model of an eye and its surrounding shapes, such as iris contour, pupil contour, and eyelids (Daugman, 2004, Gwon et al., 2013, Jafari and Ziou, 2015; Zhang, Chen, Su, & Liu, 2013). Feature-based methods identify local features in and around an eye, such as limbus, pupil, and cornea (Chen and Liu, 2015, Feng and Yuen, 2001, Kroon et al., 2009, Markuš et al., 2014, Monzo et al., 2011). Appearance-based methods model the variation of an eye by filter responses or the intensity distribution of the eye and its surrounding area (Ge et al., 2016, Qian and Xu, 2010, Valenti and Gevers, 2012, Viola and Jones, 2004). Hybrid modeling methods combine shape, feature, and appearance methods (Kim, Lee, & Kim, 2010).

In this paper, a feature-based eye detection method is applied to the gray-scaled eye images obtained by an NIR camera installed on a mobile phone. Because the pupil and iris regions are darker than their surrounding sclera region, dark region detection is employed by using the intensity information of the eye image. Feature-based methods use the local features of an eye that are robust to illumination changes (Hansen & Ji, 2010). Erroneous eye candidates may also be obtained from regions that have similar darkness to the eye such as an eyebrow or an eyeglass frame in feature-based methods. However, these can be removed by using the eye validation method proposed in this paper.

Despite much previous research on eye detection, accurate eye detection and locating are still challenging because of various conditions, such as occlusion of the eye by the eyelid, variation of head pose, changes in lighting conditions, and interference of eyeglasses. Interference of eyeglasses is a crucial problem that has not been seriously considered to date. Interference of the eyeglass frame with the eyes degrades the accuracy of eye detection. In particular, when a person is wearing eyeglasses with a thick frame, the frame around the eye region considerably affects the accuracy of eye detection. Moreover, because many adults wear eyeglasses, the interference of eyeglass frames in an eye image has to be resolved during the eye detection process. According to Prevent Blindness America and the National Eye Institute, more than 150 million Americans were wearing corrective eyewear in 2008 (Prevent Blindness America & Counsil, 2008); further, according to the Vision Council of America, approximately 63.2% of American adults were wearing eyeglasses in 2010 (Fultz & Council, 2010).

Eyeglasses may cause eye detection problems in iris recognition due to interference from the eyeglass frame and occlusion by glints on the glass surface. Both of these problems degrade eye detection accuracy. Several eye detection methods that consider the conditions when the eyes are obstructed by eyeglasses have been reported in the literature. Kim et al., 2010 proposed an eye detection method based on the eye shape feature. In their proposed method, a window mask with 3 × 3 subblocks is used to compute the difference in intensity between an eye and its surrounding regions. In addition, a priori geometric information of both eyes, such as the distance between them, is used to improve the accuracy. They conducted experiments on eyeglass wearers. However, their experiments did not consider those wearing thick or dark glass frames. In these cases, when an eyeglass frame appears in the input eye image, a part of the eyeglass frame may be included in the first row of the mask comprising 3 × 3 subblocks, causing degradation of the eye detection performance. Gwon et al. (2013) proposed an eye detection method that utilizes the AdaBoost and CAMShift algorithms to search the face area. First, they remove specular reflections that appear on the lens of the eyeglass before pupil detection. Then, they locate the center of the pupil by circular edge detection (Daugman, 2007) and binarization of the pupil area. Their proposed method removes specular reflection; however, they do not consider the problem of obstruction caused by an eyeglass frame. Valenti and Gevers (2012) proposed a method that utilizes circular symmetry based on isophote curvatures, which are curves connecting points with equal intensity (Lichtenauer, Hendriks, & Reinders, 2005). To locate the center of an eye, isophotes in which the curvedness is maximal are used to vote for a center. Consequently, after summing the votes, the point with the highest response is considered to be the location of the center of the eye. The shape of the isophotes is invariant to lighting conditions and rotation. However, due to closed eyelids and strong glints, it may detect eyeglass frame, eyebrows, and glints instead of an actual eye. Zhu et al. (2005) proposed an appearance-based eye detection method that utilizes an infrared illuminator. Specifically, their method uses the bright pupil effect caused by an IR illuminator. When an illuminator is located very close to the camera, the pupil appears bright as a result of the reflection occurring on the pupil. If another illuminator is located away from the camera, a dark pupil appears on the image. The difference image resulting from the dark eye image and the bright eye image is used by a Support Vector Machine (SVM) (Cortes & Vapnik, 1995) to detect the location of the pupil. This method is robust to reflections that appear on the lens of eyeglasses. However, if the reflections appear on the eyeglass frame, errors might occur that may result in the pupils being missed. Zhang, Chen, Yao, Li, and Zhuang (2007) proposed a method in which they detect the eye regions using AdaBoost (Viola & Jones, 2004) and locate the center of the pupil by Fast Radial Symmetry, which detects the points with high radial symmetry in intensity. Their results suggest that redial symmetry transform can avoid the interference of eyeglass frames in the eye detection stage. However, AdaBoost may detect the eye region inaccurately because of the interference of eyeglass frames in the eye-region detection stage. Several researches have considered the interference of specular reflections caused by eyeglasses. However, no research focusing on an eye detection method that avoids the interference of eyeglass frames has been discovered to date.

In this paper, an eye detection method that is robust to interference from eyeglasses is proposed for iris recognition. Eye detection is defined here as the process used to determine both the existence and location of an eye in the region-of-interest image (Hansen & Ji, 2010). The proposed method consists of two stages: 1) eye candidate generation; and 2) eye validation. In the eye candidate generation stage, eye candidates are generated by computing the difference in intensity between the iris region and its surrounding regions using multi-scale window masks which fit within an eyeglass frame in order not to include the eyeglass frame in the mask area when locating the eye. False findings that are produced by the masks are removed in the eye validation process. In the eye validation stage, two methods are employed to ascertain if a pupil region is in the mask area found in the previous stage. Firstly, a pupil region is detected by searching the glint of the NIR illuminator that normally appears inside the pupil region. In mobile iris recognition, because the distance between the camera and illuminators is small, the glint of NIR illuminators typically appears inside the pupil region. This fact is used in the proposed eye validation method. If this method fails to find the pupil of the eye, then the next method, which computes the intensity difference between the pupil region and its surrounding regions using another much smaller 3 × 3 window mask than the one used for finding the eye candidates, is applied to find the pupil of the eye. The main contributions made in this paper are as follows:

• A fast eye detection algorithm robust to eyeglasses is proposed for implementation on mobile devices

  • Multi-scale window masks consisting of 2 × 3 subblocks are proposed to generate eye candidates in the search for an eye. The location of the eye is quickly and accurately obtained while avoiding interference from any eyeglass frame appearing in the eye image.

  • An eye validation approach that employs two methods applied in sequence to select the true eye image among the candidates produced during eye candidate generation is proposed. This step enhances eye detection accuracy.

• Analysis of the eye images of various races acquired by an NIR camera is provided to formulate an eye candidate generation threshold for different races.

• A glint detection method is proposed to detect a glint that appears inside a pupil and a larger glint on the glass surface.

The remainder of this paper is organized as follows: details of the proposed eye detection method are given in Section 2. Experimental results and performance analysis are presented in Section 3. Finally, concluding remarks are given in Section 4.