Elsevier

Pattern Recognition

Volume 47, Issue 4, April 2014, Pages 1573-1585
Pattern Recognition

Face detection based on skin color likelihood

https://doi.org/10.1016/j.patcog.2013.11.005Get rights and content

Highlights

  • We propose a boosting-based face detection method using skin likelihood.

  • Our method emphasizes skin color while deemphasizing non-skin color.

  • Good tolerance to severe face pose variation is obtained.

  • The proposed method shows robustness against complex background.

Abstract

We propose a face detection method based on skin color likelihood via a boosting algorithm which emphasizes skin color information while deemphasizing non-skin color information. A stochastic model is adapted to compute the similarity between a color region and the skin color. Both Haar-like features and Local Binary Pattern (LBP) features are utilized to build a cascaded classifier. The boosted classifier is implemented based on skin color emphasis to localize the face region from a color image. Based on our experiments, the proposed method shows good tolerance to face pose variation and complex background with significant improvements over classical boosting-based classifiers in terms of total error rate performance.

Introduction

Human face detection is among the most important topics in biometric research since it has a broad range of applications. Detection of face is often performed prior to recognition and tracking in biometric and surveillance systems. A variety of techniques have been proposed for face detection in the literature where they can be generally classified into the following categories [1]: knowledge-based methods, invariant feature methods, template matching methods and appearance-based methods.

Knowledge-based methods are rule-based methods which encode human knowledge of what constitutes a typical face. Usually, some rules are designed to capture the relationships among the facial components. Invariant feature methods adopt features such as facial components, texture, skin color and a multiple of these features for face detection. These methods aim to find common structural features which exist among faces under different ambient conditions. Template matching methods store several standard patterns of a face to describe the face either as a whole or as separate facial components. Appearance-based methods learn a model or a group of features from a set of training images to capture the representative variability of facial appearance.

Most of the face detection techniques incur a large number of false rejections due to severe face pose variation and false acceptances due to complex background. To address these issues, we propose a face detection method based on skin color emphasis and iterative boosting to selectively highlight the skin color information and deemphasize background information. Unlike other boosting-based methods using skin color, our method uses neither parametric curve fitting nor morphological operators. Skin color is used for skin color emphasis rather than skin color segmentation.

Our main contributions of this work include the tolerance of proposed system to face rotation and complex background. The boosted classifier reacts less sensitively to face pose variation as it concentrates on probabilistic distribution of facial skin color rather than the details of facial components in gray-level brightness. Also, non-skin color information including background is significantly reduced, so that skin color likelihood can be discriminatively learned.

The organization of this paper is as follows. Section 2 provides a review on related works in face detection using skin color information. Section 3 describes our proposed method in detail. Section 4 presents the experimental results of our method on several face databases. Finally, our conclusion is given in Section 5.

Section snippets

Related works

Many face detection methods based on a face model have been proposed to cope with varying conditions including face rotation and complex background. Wang and Yuan [2] proposed a human face detection from color images under complex conditions including arbitrary image background. They used an evolutionary computation technique to cluster skin-like color pixels and segment each face-like region. After the face-like regions are located, the wavelet decomposition is applied to each face-like region

Skin likelihood

The YCbCr space can be easily obtained from the RGB space by a simple matrix operation. Eq. (1) shows the actual conversion from RGB to YCbCr according to [15](YCbCr)=(0.2990.5870.1140.1680.3310.50.50.4180.081)(RGB)

The YCbCr space is perceptually uniform, and it separates luminance and chrominance presenting compactness of the skin distribution cluster [4] as shown in Fig. 1a. Human skin forms a relatively tight cluster in color space even when different races are considered [16], [17],

Experimental setup

To evaluate our proposed method, we used color face images under complex background and largely varying pose of face in yaw, roll and pitch directions. In addition, varying lighting condition, different races, facial expression and slight changes in appearance such as moustache or glasses are also basically considered. We used 393 images of Pointing’04 database [23] (Fig. 6) and 1200 images of IMM database [24] (Fig. 7) to test our method against face pose variation in the presence of varying

Conclusion

We proposed a boosting-based face detection method based on the likelihood of skin color in this paper. Our method emphasizes skin color information while simultaneously deemphasizes non-skin color information. Skin color emphasis could be well combined with iterative boosting algorithm. The proposed method shows improvement over those conventional methods of [9], [10], [11], [12], [13], [14] against severely varying face pose and complex background. Our proposed method substantially reduces

Conflict of interest statement

None declared.

Acknowledgment

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (NRF-2011-0016302).

Yuseok Ban received his B.S. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. Currently, he is a candidate of Ph.D. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. His research interests include pattern recognition, biometrics and computer vision.

References (31)

  • Iyad Aldasouqi et al.

    Smart human face detection system

    Int. J. Comput.

    (2011)
  • Sanjay Kr. Singh et al.

    A robust skin color based face detection algorithm

    Tamkang J. Sci. Eng.

    (2003)
  • Vladimir Vezhnevets, Vassili Sazonov, Alla Andereeva, A survey on pixel-based skin color detection techniques, in:...
  • P. Viola, M. Jones, Rapid object detection using boosted cascade of simple features, in: Proceedings of the 2001 IEEE...
  • R. Lienhart, J. Maydt. An extended set of Haar-like features for rapid object detection, in: Proceedings of ICIP, 2002,...
  • Cited by (79)

    • Skin detection and lightweight encryption for privacy protection in real-time surveillance applications

      2020, Image and Vision Computing
      Citation Excerpt :

      As a result, in this paper, color-based segmentation for skin detection is considered as the most appropriate method of detection along with privacy-protection measures aimed at real-time surveillance applications. It is a well-established fact [1,2,4,8] that color-based segmentation is a powerful approach for skin classification but, due to its simpler implementation, it is considered to lack the robustness of advanced machine learning, especially CNN, approaches. However, simple approaches, along with privacy protection arising from encryption are suitable for real-time surveillance applications and, particularly emerging IOMT applications.

    • Faceboxes: A CPU real-time and accurate unconstrained face detector

      2019, Neurocomputing
      Citation Excerpt :

      The pioneering work of Viola and Jones [1] is the first major breakthrough, which uses Adaboost with Haar features to train a series of cascaded classifiers to detect face and achieves satisfactory accuracy with high efficiency. Since then, several methods focus on designing new local features [44–48], new boosting algorithms [49–52] and new cascade structures [53–57]. DPM [4] is another popular method in face detection [58–61] by using mixtures of multi-scale deformable part models to represent multi-view faces.

    • Multiple feature descriptors based model for individual identification in group photos

      2019, Journal of King Saud University - Computer and Information Sciences
      Citation Excerpt :

      The localization results were obtained by the author against same measures. The comparative observation against the results derived by Ban et al. (2014) is here provided hereunder. Fig. 14 is showing the comparative evaluation of facial localization under FAR parameter.

    • Manga face detection based on deep neural networks fusing global and local information

      2019, Pattern Recognition
      Citation Excerpt :

      One viewpoint to handling the manga face detection problem is viewing it as a spacial object detection problem. Based on various visual features like contour, shape [7], texture [8], and color contrast [9–11], tremendous approaches have been proposed to do object matching or object detection. Thanks to the development of deep neural networks, recently very impressive object detection and recognition models have been proposed for natural images/videos [12,13].

    • Robust in-plane and out-of-plane face detection algorithm using frontal face detector and symmetry extension

      2018, Image and Vision Computing
      Citation Excerpt :

      One of the disadvantages of the neural network, however, is that it needs to collect a large amount of training data and requires very long training time. Other pixel-based algorithms use the feature of the skin color in different color spaces to detect the face [11–14]. These methods are very fast and robust to face rotation but are intolerant of environmental changes such as the lighting effect.

    View all citing articles on Scopus

    Yuseok Ban received his B.S. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. Currently, he is a candidate of Ph.D. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. His research interests include pattern recognition, biometrics and computer vision.

    Sang-Ki kim received his B.S. and Ph.D. degrees in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea in 2004 and 2011 respectively. He is currently a senior research engineer at LG Electronics. His research interests are in the fields of pattern recognition and computer vision with special focus on face and gesture recognition.

    Sooyeon Kim received her B.S. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. Currently, she is a candidate of Ph.D. degree in Electrical and Electronic Engineering from Yonsei University, Seoul, Korea. Her research interests include face and gesture recognition.

    Kar-Ann Toh is a full time professor in the School of Electrical and Electronic Engineering at Yonsei University, South Korea. He received the Ph.D. degree from Nanyang Technological University (NTU), Singapore. He worked for 2 years in the aerospace industry prior to his post-doctoral appointments at research centres in NTU from 1998 to 2002. He was affiliated with Institute for Infocomm Research in Singapore from 2002 to 2005 prior to his current appointment in Korea. His research interests include biometrics, pattern classification, optimization and neural networks. He is a co-inventor of a US patent and has made several PCT filings related to biometric applications. Besides being an active member in publications, Dr. Toh has served as a member of technical program committee for international conferences related to biometrics and artificial intelligence. He is currently an associate editor of Pattern Recognition Letters and a senior member of the IEEE.

    Sangyoun Lee received his B.S. and M.S. degrees in Electronic Engineering from Yonsei University, Seoul, South Korea in 1987, 1989 respectively. He received his Ph.D. degree in Electrical and Computer Engineering from Georgia Tech., Atlanta, Georgia, in 1999. He was a Senior Researcher in Korea Telecom from 1989 to 2004. He is now a faulty member of the School of Electrical and Electronic Engineering, Yonsei University, Korea. His research interests include pattern recognition, computer vision, video coding and biometrics.

    View full text