Elsevier

Pattern Recognition

Volume 35, Issue 7, July 2002, Pages 1583-1596
Pattern Recognition

A statistic approach to the detection of human faces in color nature scene

https://doi.org/10.1016/S0031-3203(01)00146-7Get rights and content

Abstract

In this paper, a novel algorithm for oriental face detection is presented to locate multiple faces in color scenery images. A binary skin color map is first obtained by applying the skin/non-skin color classification algorithm. Then, color regions corresponding to the facial and non-facial areas in the color map are separated with a clustering-based splitting algorithm. Thereafter, an elliptic face model is devised to crop the real human faces through the shape location procedure. Last, local thresholding technique and a statistic-based verification procedure are utilized to confirm the human faces. The proposed detection algorithm combines both the color and shape properties of faces. In this work, the color span of human face can be expanded as wilder as possible to cover different faces by using the clustering-based splitting algorithm. Experimental results reveal the feasibility of our proposed approach in solving face detection problem.

Introduction

Human face detection is the first step in face recognition systems, such as criminal identification and security system. Besides, the detection algorithm can also be applied in a content-based image retrieval system (CBIR) to search the images that contain human beings or computer-aided equipment for handicapped people to command the computer. Unless the faces in the image could be detected successfully, the recognition system will not work at all. Hence, the development of a reliable face detection algorithm is a prerequisite for a successful face recognition system. Over the past years, several researchers proposed the face detection and analysis methods in a simple background image [1], [2], [3], [4]. Besides, other researchers proposed the methods in a complex background [5], [6], [7], [8], [9], [10]. Initially, most of the aforementioned methods [1], [2], [3], [4], [5], [6] utilized the black-and-white or gray level images in their works. Recently, face detection in color images [7], [8], [9], [10] has attracted more attention due to the uplift of chip manufacture technology, the improvement of compression techniques and the construction of broad band network. These progresses have made the requirements to manipulate or transmit color images easier to meet. Furthermore, most of the past works detected the faces in the mug-shot or head-and-shoulder images, which made the faces easily and accurately to be segmented from background. However, they are unrealistic and inflexible for practical applications.

As we know, a color image contains more information than a gray level or black-and-white image. In addition, there are many color systems that can be utilized for users’ favorites, such as, RGB, HSI, XYZ, YIQ, and YES. During the recent years, human face detection algorithm based on color images has attracted the attention of many researchers. Lee et al. [7] proposed a two-stage location method. The first step is to detect and segment the faces from the background by applying the motion information. Next, the hue axis of HSI color system is adopted to separate the mouth from other skin regions of the face candidate. Accordingly, the eyes and eyebrows can be extracted by utilizing the knowledge-based location procedure. The limitation of their method is that it must keep the head moving in front of the camera. Besides, they needed the illuminated light source to keep the lighting distributed uniformly on the face. Furthermore, it seems to contain only the head-and-shoulder images with fixed background in their experiments. Dai and Nakano [8] proposed a model-based algorithm to detect the positions of faces in an image. They first converted the RGB color image into a YIQ image. Then, the face-texture model based on SGLD matrix was applied to extract the faces in the I component of the YIQ image. This algorithm can successfully detect the faces in the scenery pictures indoors and outdoors. However, the size of faces can only be varied from 16×20 to 20×26 pixels because the size of scanning window is fixed.

Besides, the CIE-xyz color system and neural network were first utilized by Chen and Chiang [9] to classify the eyebrows, lip, skin and non-skin color regions in a color image. Then, the lip in the face candidate was confirmed by applying the energy-thresholding method. Finally, the face was segmented according to the location of the confirmed lip. The images they used were divided into three groups, the images of Group 1 were taken from digital camera under different lighting conditions, images of Group 2 were scanned from magazines or newspapers, and images of Group 3 were obtained from video tape or network. Wherein, the performance of their proposed method is not bad. However, it seems so easy to directly classify the eyebrows, lip and skin color regions by using only a color classification algorithm. Identically, Saber and Tekalp [10] directly utilized the YES color system to classify the skin and non-skin color regions in the image. Then, the eyes, nose and mouth on the face were extracted by applying the symmetry-based cost functions. Furthermore, the color segmentation for skin color was also used in the works of Chang et al. [11] and Chen et al. [12].

Almost all the aforementioned detection methods were proposed to directly segment the faces embedded in the images by utilizing only a color classification algorithm. However, this will lead to the miss of some real faces because the color of face spans too wide to be covered by only some narrow color zones. In this paper, we propose a skin color classification method with wider color span combining with a clustering-based splitting algorithm to remedy this problem. Besides, in order to increase the flexibility of face detection algorithm for different applications, we develop the algorithm to detect multiple and frontal oriental faces in color scenery images with complex background and different scales, lighting conditions.

In our approach, the color classification method is first applied to classify the skin-like and non-skin-like color regions in the original image. Then, a clustering-based splitting algorithm is devised to separate the real human faces and other skin color regions. Thereafter, the real faces are located and labeled as “candidate face” by applying the model-based face location algorithm. Finally, the faces are verified by using the local thresholding technique and statistic-based verification procedure in the face verification module. Fig. 1 shows the system block diagram of our proposed system.

The rest of this paper is organized as follows. Section 2 introduces the skin/non-skin color classification algorithm. The clustering-based splitting algorithm is presented in Section 3. Section 4 addresses the model-based face location algorithm. The face verification procedure is described in Section 5. Section 6 demonstrates the experimental results. Finally, conclusions are given in Section 7.

Section snippets

Color classification

It is generally agreed that there is no single color system which is suitable for all color images [13]. Hence, it is unnecessary to insist on the adoption of specific color system to be used in the color classification algorithm. However, RGB and XYZ color systems are sensitive to the intensity variations. Hence, they are not suitable for our application in developing a face detection algorithm with wider color span. In this paper, the HSI color system is adopted to design the color

The clustering-based splitting algorithm

In this paper, the specified zone defined in the previous section is selected as wide as possible to cover all the skin colors. Consequently, the color that does not belong to a face will also be covered in this zone. Hence, a splitting algorithm must be applied to separate the facial and non-facial color regions in the classified HSI image. It is an important contribution of our work. In the same way, Chai and Ngan [14] proposed morphological operations, such as dilation and erosion, to

Model-based face location

A face model is devised here to locate the real human faces by comparing the shape of face model and that of each color region obtained in the previous section. Firstly, the mean vector (μ) and covariance matrix (C) of the tested color region are calculated as follows:μ=1Nj=1NXj,C=1Nj=1NXjXjT−μμT,where N is the number of pixels in the tested color region, Xj is the coordinate vector of the jth pixel, and superscript T denotes the transpose of a vector. These equations point out that the loci

Face verification

The candidate faces cropped from the original RGB image are further verified in this section by applying the local thresholding technique and a statistic-based verification procedure. The local thresholding technique is first applied to binarize each candidate face image. Then, the facial features (eyes and mouth) are extracted by utilizing some geometric rules. Finally, a statistic-based verification procedure is executed to verify the real human faces.

Experimental results

Most of the past works [1], [2], [3], [4], [5], [7], [10] conducted the experiments on the mug-shot or head-and-shoulder images. In this paper, experiments were conducted on color scenery images to verify the validity of our proposed algorithms. Firstly, a database containing 60RGB color images taken by Kodak DC210 digital camera is adopted to test the proposed detection algorithm. Each image contains at least one human face and there are totally 72 human faces in the database. Besides,

Conclusions

In this paper, a novel face detection algorithm combining the color and shape properties of face is proposed. The color classification algorithm is first applied to classify the skin and non-skin pixels in a nature scenery image. Thereafter, a clustering-based splitting algorithm is used to separate the real human faces from other skin color regions. Then, a face model is designed to test the shape of a real face. Finally, a statistic-based verification procedure is utilized to confirm the

Summary

In this paper, a novel algorithm for oriental face detection is presented to locate multiple faces in color scenery images. The proposed detection algorithm combines the color and shape properties of faces. Experimental results reveal the feasibility of our proposed approach in solving face detection problem.

During the past years, almost all detection methods were proposed to directly segment the faces embedded in the images by utilizing only a color classification algorithm. However, this will

About the Author—ING-SHEEN HSIEH was born on 5 November 1957 in Changhua, Taiwan, Republic of China. He received his B.S. and M.S. degrees in electrical engineering from National Cheng-Kung University, Taiwan, in 1980 and 1987, respectively. From 1982 to 1987, he was a research assistant in the Chung-Sun Institute of Science and Technology (CSIST), Taiwan, where he has been an assistant researcher since 1987. In 2000, he received his Ph.D degree in Institute of Computer Science and Information

References (17)

There are more references available in the full text version of this article.

Cited by (64)

  • A skin detection approach based on the Dempster-Shafer theory of evidence

    2012, International Journal of Approximate Reasoning
  • An automated anti-pornography system using a skin detector based on artificial intelligence: A review

    2013, International Journal of Pattern Recognition and Artificial Intelligence
  • Multi-task Learning Based Skin Segmentation

    2023, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
  • A person's face complex image processing in the visible and infrared ranges for current psycho-emotional and functional state monitoring systems

    2018, Proceedings of the 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering, ElConRus 2018
View all citing articles on Scopus

About the Author—ING-SHEEN HSIEH was born on 5 November 1957 in Changhua, Taiwan, Republic of China. He received his B.S. and M.S. degrees in electrical engineering from National Cheng-Kung University, Taiwan, in 1980 and 1987, respectively. From 1982 to 1987, he was a research assistant in the Chung-Sun Institute of Science and Technology (CSIST), Taiwan, where he has been an assistant researcher since 1987. In 2000, he received his Ph.D degree in Institute of Computer Science and Information Engineering from National Central University. His current research interests include pattern recognition and image analysis.

About the Author—KUO-CHIN FAN was born on 21 June 1959 in Hsinchu, Taiwan, Republic of China. He received his B.S. degree from National Tsing-Hua University, Taiwan, in 1981, and the M.S. and Ph.D. degrees from the University of Florida, Gainesville, in 1985 and 1989, respectively, all in electrical engineering. In 1983, he was a Computer Engineer with the Electronic Research and Service Organization (ERSO), Taiwan. From 1984 to 1989 he was a Research Assistant with the Center for Information Research, University of Florida. In 1989, he joined the Department of Computer Science and Information Engineering, National Central University, Taiwan, where he became Professor in 1994. He is currently the Chairman of the Computer Center, National Central University. His current research interests include pattern recognition, image processing, computer vision and neural networks. Dr. Fan is a member of SPIE.

About the Author—CHIUNHSIUN LIN was born on 5 December 1961 in Chia-yi, Taiwan, Republic of China. He received his M.S. degrees in Computer Science from DePaul University, Chicago in 1993. He is an assistant researcher in Committee for Planning and Organizing the National Taipei University since 1995. In 1996, he entered the Institute of Computer Science and Information Engineering at National Central University working toward his Ph.D. degree. His current research interests include pattern recognition, face detection, face recognition, image processing, and image analysis.

View full text