Hand shape recognition based on coherent distance shape contexts

Abstract

In this paper, we propose a novel hand shape recognition method named as Coherent Distance Shape Contexts (CDSC), which is based on two classical shape representations, i.e., Shape Contexts (SC) and Inner-distance Shape Contexts (IDSC). CDSC has good ability to capture discriminative features from hand shape and can well deal with the inexact correspondence problem of hand landmark points. Particularly, it can extract features mainly from the contour of fingers. Thus, it is very robust to different hand poses or elastic deformations of finger valleys. In order to verify the effectiveness of CDSC, we create a new hand image database containing 4000 grayscale left hand images of 200 subjects, on which CDSC has achieved the accurate identification rate of 99.60% for identification and the Equal Error Rate of 0.9% for verification, which are comparable with the state-of-the-art hand shape recognition methods.

Introduction

In information and network society, there are many occasions in which the personal authentication is required, e.g., access control to a building, information security in a computer or smart-phone, visitor management, electronic payment, etc. There is no doubt that biometrics is one of the most important and effective solutions for this task. Generally, biometrics is a field of technology that uses automated methods for identifying or verifying a person based on a physiological or behavioral trait [1]. In real applications, the traits that are commonly measured in different systems are face, fingerprints, iris, hand shape, palmprint, hand vein, ear, gait, voice, etc.

Among all kinds of biometrics techniques, hand shape biometrics has drawn wide attention from researchers, and has been investigated for a long time [2]. As often noted in the literatures, hand shape biometrics is attractive due to its following advantages [2], [3]. (1) It is user-friendly since hand shape can be captured in a relatively convenient, non-intrusive manner by using inexpensive sensors. (2) Other hand features such as palmprint, palm vein and knuckleprint can be easily integrated to an existing hand shape based biometric system to form a more robust multimodal biometrics system. (3) It is more acceptable to the public mainly because it lacks criminal connotation. (4) It is less prone to disturbances and more robust to environmental conditions and to individual anomalies.

There have been a number of previous studies for hand shape biometrics. In the early study of this technique, most commercial systems [4] and some of the research systems [5], [6], [7], [8] required a platform with pegs to guide the placement of the user's hand. This manner, however, is not user-friendly and is unreliable since pegs often cause heavy deformation of the hand silhouette. Recently, some more user-friendly systems use non-contact image acquisition setups to capture hand images [9], [10], [11], [12]. Obviously, such setups would introduce more variations of hand pose, which will result in poor recognition performance. In order to make a trade-off between the user-friendliness and reliability, most systems capture hand images by using peg-free and contact manner [3], [13], [14], [15], [16], [17], [18], [19].

Generally speaking, the approaches of peg-free and contact hand shape recognition could be divided into three categories, i.e., the contour-based, the region-based, and the geometry-based, respectively. The contour-based approaches usually extract features from the whole shape contour, which consists of discrete points. G. Amayeh et al. [13] proposed using high-order Zernike moments, which are powerful shape descriptors, to capture hand shape features, and applied an efficient algorithm to deal with several practical issues of the moments including computational cost and lack of accuracy. H. Dutagaci et al. [17] proposed a normalization algorithm to adjust the fingers to standard positions, and extracted shape features by applying Linear Discriminant Analysis (LDA) on Discrete Fourier Transform (DFT) coefficients of the contour. The region-based methods usually extract features from the binary images, where the shape is represented as a connected region. E. Yoruk et al. [3], [14] proposed a global hand shape biometrics approach, which is pivoted on the normalization of the deformable hand shape and extract independent component analysis (ICA) features to compare the hand region images. They notice that the thumb can potentially arrive in a curved posture even after the whole hand exerting on the platen, which would cause the normalization of the hand shape unreliable in a real application environment. So it is believed that omitting the thumb is a good choice to ensure a robust system [2]. J. M. Ramirez-Cortes et al. [19] proposed an hand shape identification approach using the morphological pattern spectrum. In this method, the structuring element of the morphological operations is given by a disk with an incremental radius of one pixel per iteration, and a spectrum-based feature vector is obtained from the hand shape image for recognition. The geometry-based methods extract physical or abstract shape geometry from the shape. In geometry-based methods, a lot of landmark points (e.g., finger tips or valleys) would be determined before the feature extraction. G. Fouquier [15] presented an authentication system based on simple finger geometric features. In this method, for each finger, each boundary point is projected on the finger major axis. The lengths of the projected segments are computed and a histogram of these lengths is considered as features for recognition. M. Adan et al. [16] proposed a Natural Reference System (NRS) of the hand's natural layout and used it to extract the polar representation of the hand's contour for comparison. Under the NRS some abstract geometric features are extracted and used for recognition, such as the lengths between landmark points of NRS.

Though the existing hand shape recognition approaches have achieved promising recognition performances, they still have some shortcomings. For example, most contour based approaches extracted features from whole contour of a hand for recognition. Parts of contour around finger valleys, however, may have obvious changes caused by different hand poses, which is not reliable for robust recognition. Thus, it is difficult to obtain satisfying recognition results for those contour-based approaches in which the negative effect of changes of finger valleys has not been fully taken into account. In order to remove this negative effect, a normalization procedure has been proposed to adjust the fingers to some standard positions [14]. This procedure, however, would cost extra processing time and is not available when the fingers are at extreme positions, such as the hands are fully extended. For those geometry-based methods, the geometry features are usually extracted from some specific positions of the hands. Obviously, this manner could not capture enough discriminative information [17]. More importantly, it is difficult to robustly extract those landmark points when the fingers are at different positions. Exactly speaking, some landmark points of a hand could be easily determined, but they may not exactly correspond to landmark points of another hand due to the position variations of fingers. This inexact correspondence problem would result in unreliable features, therefore, cause recognition errors.

In the past decade, many shape matching algorithms have been proposed [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], and they are applied to different shape-based applications. In this paper, we propose a novel contour-based hand shape recognition method, which is based on two classical shape matching methods, namely Shape Contexts (SC) [20] and Inner-distance Shape Contexts (IDSC) [32]. It should be noted that both SC and IDSC have not been exploited for the application of hand shape recognition so far. Both of them uniformly extract points along the contour, and utilize the global and local geometry simultaneously. The features extracted by SC and IDSC not only contain the geometry features in common geometry-based methods, but also take into account the distributions of those features, which intrinsically capture more discriminative information of the shapes than traditional geometry-base methods. Meanwhile, the matching procedure of SC and IDSC is based on the method of dynamic programming (DP), which could find the exact correspondences of contour points between two hands and relieve the inexact correspondence problem in traditional geometry-based methods. However, original SC and IDSC have some limits. For example, using Euclidean distances between contour points, SC is quite sensitive to different hand poses. Though the inner-distance introduced in IDSC largely solves the hand pose problem, it is still affected by elastic deformations of finger valleys. Therefore, they are not suitable for the hand shape biometrics. In order to effectively overcome these limits, we fully consider the prior knowledge, and then propose a new method by modifying many details of SC and IDSC. For example, SC uses the Euclidean distance and IDSC uses the inner-distance to construct features, both of which suffer from the unreliable parts of the contour. To address this problem, we introduce a new distance measure, namely coherent distance, which could capture the robust distance features of a shape and is more suitable for hand shape recognition.

The main contributions of our method are as follows: First, we prove that the parts of contour around finger valleys are not reliable for robust hand shape recognition since they have obvious changes caused by different hand poses. Second, we propose a novel hand shape recognition method that takes into account the negative effect of changes of finger valleys. Meanwhile, this method could also well deal with the inexact correspondence problem and capture enough discriminative information of the hand shapes, therefore, can achieve good recognition performance. Third, considering the practical issues including computational cost and robustness, a speed-up procedure and a fusion procedure are proposed.

The rest of this paper is organized as follows: Section 2 describes the image collection device and the preprocessing algorithm. Section 3 makes a brief review of the methods of SC and IDSC. Section 4 presents the details of the proposed method. Section 5 introduces the new hand image database and reports the experimental results, and Section 6 concludes the whole paper.