Computerized facial diagnosis using both color and texture features

doi:10.1016/j.ins.2012.09.011

Information Sciences

Volume 221, 1 February 2013, Pages 49-59

https://doi.org/10.1016/j.ins.2012.09.011 Get rights and content

Abstract

Facial diagnosis is an important diagnostic tool, and has been practiced by various traditional medicines for thousands of years. However, due to its qualitative and subjective nature, it cannot be accepted in mainstream medicine. To circumvent these issues, computerized facial diagnosis using color and texture features are extracted from facial blocks representing a facial image. A facial color gamut is constructed and six centroids located to help calculate the facial color feature vector. As for the texture feature, a 2-dimensional Gabor filter with various scales and orientations are applied. Both features are combined to diagnosis the face. The experimental results were carried out on a large dataset consisting of 142 Health and 1038 Disease samples. Using both extracted features facial gloss was first detected and employed to distinguish Health and Disease samples with an average accuracy of 99.83%. Illnesses in Disease were also separated by the analysis of each facial block. The best result was achieved using all facial blocks, which successfully classified (>71%) six illnesses.

Introduction

Traditional medicines view facial diagnosis as a staple and is used by many practitioners [1], [2], [3], [4]. In Traditional Chinese Medicine (TCM) [1], [2], [3], [4] for example, the belief is that the cause, symptoms and the origin of the disease can be reflected on one’s face through color and texture changes. However, the diagnostic result is usually based on years of experience (made by the practitioner) and can be thought of as subjective or qualitative. To eliminate this bias, computerized facial diagnosis based on quantitative feature extraction and analysis can be established.

Only a few papers on this topic were found in a literature review. [5] first used a chamber with digital camera and LED light to capture a precise facial image from patients using standardized illumination conditions. Then, a five color scale was setup to measure facial complexion in clinical diagnosis with the help of TCM doctors. A method was proposed in [6] to diagnosis heart disease through facial color. The central part of the forehead and lips were extracted and analyzed as they portray heart disease according to oriental medicine. Five facial blocks were taken from images in [7] to diagnosis hepatitis, with the average RGB pixel intensities of all blocks used as a feature. Using 116 patients suffering from two types of liver disease and 29 healthy volunteers, the proposed work achieved an average accuracy of 73.6% at separating the three classes. In [6] only heart disease was diagnosed and based on oriental medicine. The color complexion of [5] was defined in the TCM framework and marked by its practitioners. Both these methods introduce a form of bias in the diagnostic result. [7] approached facial diagnosis using facial colors by removing the subjective and qualitative properties associated with TCM. However, the facial images were not color corrected to ensure uniformity in the analysis, and only one disease, hepatitis was identified. In all applications the experimental results were carried out on smaller databases with a single disease diagnosed. Also, none made use of facial texture to compliment color information.

To resolve these issues, this paper utilizes both color and texture features extracted from facial blocks under a large database setting for diagnosis. Facial images are first captured with a specially designed device and calibrated to ensure consistency in feature extraction and analysis. Four facial blocks, one on the forehead and nose, and two below the left and right eye are used to represent a face. A facial color gamut is constructed with six color centroids used to compute the color feature vector. The Gabor filter [8], [9] with various scales and orientations are applied to the blocks and form the texture feature. Experimental results were carried out on a large scale dataset consisting of 143 Health and 1380 Disease samples taken from Guangdong Provincial Hospital of Traditional Chinese Medicine, Guangdong, China. The Disease class was composed of 6 specific diseases (with at least 10 samples in each group) and one sizeable miscellaneous group (made-up of various illnesses). Classification was performed between Health vs. Disease using facial gloss [10], [11], [12], and Disease vs. Disease by both features and a combination of facial blocks.

The rest of this paper is organized as follows. Section 2 discusses the facial image acquisition device, dataset used and facial block definition. In Section 3 facial feature extraction using both color and texture are described in detail. Facial gloss is detected in Section 4 and applied at distinguishing samples in the two classes of Health and Disease. Following this, the classification between diseases in the Disease class at first using each block, and then a combination of blocks is discussed in Section 5. Finally, concluding remarks are made in Section 6.

Section snippets

Facial images and dataset

In the following, the facial image acquisition device is described in Section 2.1 along with image calibration to ensure consistency. Afterwards, the facial image dataset is introduced in Section 2.2, and followed by a discussion on facial image block definition (Section 2.3).

Facial features extraction

Facial feature extraction is described in this section. First, color feature extraction using the facial color gamut is presented in Section 3.1 and followed by six centroids representing the main colors of facial blocks. The six centroids are then used to calculate a facial color vector for each block. In Section 3.2 texture feature extraction is discussed by applying the Gabor filter.

Healthy classification using facial gloss

In the following, facial gloss is defined using both color and texture features described above for healthy classification. Facial gloss on the skin can be described as reflective, shiny and smooth [10], [11], [12]. It is deeply rooted in TCM, and its existence indicates a patient as being healthy. However, no quantitative evidence in the literature to back up this claim was found. For this reason, facial gloss is an ideal candidate to distinguish between Health and Disease samples. Applying

Facial block-based disease analysis

Facial block-based disease analysis is carried out in two steps. In the initial step, each facial block is analyzed independently (Section 5.1). Following this, the optimal combination of block(s) is determined in Section 5.2 by computing the number of illnesses each group can detect, and selecting the group that detected the most.

Conclusions

Using both color and texture features extracted from facial blocks, computerized facial diagnosis is possible, achieving a good outcome according to experimental results on the Guangdong dataset. Samples in Health and Disease were separated with an average accuracy of 99.83% via facial gloss found in blocks A–C. Further experiments showed block A can be used independently to detect facial gloss while attaining the same accuracy. To examine whether illnesses in Disease can be distinguished from

Acknowledgements

The work is partially supported by the GRF fund from the HKSAR Government, the central fund from Hong Kong Polytechnic University, and the NSFC Oversea fund (61020106004), China.

References (16)

G. Maciocia
The Foundations of Chinese Medicine
(1989)
M. Porkert
Theoretical Foundations of Chinese Medicine: Systems of Correspondence
(1978)
L. Bridges
Face Reading in Chinese Medicine
(2004)
B. Liu et al.
Inspection of Face and Body for Diagnosis of Diseases
(2002)
X. Li et al.
Computer-aided disease diagnosis system in TCM based on facial image analysis
International Journal of Functional Informatics and Personalized Medicine
(2009)
B.H. Kim, S.H. Lee, D.U. Cho, S. Oh, A proposal of heart diseases diagnosis method using analysis of face color, in:...
M. Liu, and Z. Guo, “Hepatitis diagnosis using facial color image,” in Proc. International Conference on Medical...
R.O. Duda et al.
Pattern Classification
(2000)

There are more references available in the full text version of this article.

Cited by (20)

Designing intelligent self-checkup based technologies for everyday healthy living
2022, International Journal of Human Computer Studies
Recent years have seen increasing interest in health promotion or disease prevention, rather than disease treatment/management, as a more cost-effective approach towards healthcare. Technologies designed to help build healthy habits, or lifestyle technologies, have important roles to play in that respect. However, so far, they are mainly explored without considering individual health statuses, which may lead to people’s excessive pursuit of quantified goals and cause negative health outcomes. In addition, while AI-powered health checkup technologies have become available for consumers to assess health on their own, they are mainly disease-oriented, and their potential for preventive health – everyday health assessment and early interventions through lifestyle changes – has been largely under-explored. This paper attempts to combine intelligent self-checkup and personalized lifestyle technologies so as to provide suggestions based on personal health status, as a potentially more effective approach to support preventive health in everyday life, or everyday healthy living as we call it in this paper. In particular, we present the iterative design process of a mobile application which supports AI-powered everyday health assessment based on face examination and adaptive inquiry, and provides personalized and adaptive lifestyle advice. Design improvements were made based on the first pilot study, and a 4-week field trial of the final design prototype reveals a number of design implications for intelligent health technologies to better support everyday healthy living.
Computational Traditional Chinese Medicine diagnosis: A literature survey
2021, Computers in Biology and Medicine
Citation Excerpt :
Wang et al. also evaluated computerized detection for icterohepatitis and severe hepatitis [51]. In their work, they used a similar digital facial acquisition device like [49] containing a digital video camera and fluorescent lamps, for collecting normal healthy samples (129), icterohepatitis samples (69), and severe hepatitis samples (111), correspondingly. From this, they applied quantitative chromatic features, followed by feature selection for facial images, and further processed with the SVM model to produce the final results (overall diagnostic accuracy is higher than 80%).
Traditional Chinese Medicine (TCM) diagnosis is based on the theoretical principles and knowledge, where it is steeped in thousands of years of history to diagnose various types of diseases and syndromes. It can be generally divided into four main diagnostic approaches: 1. Inspection, 2. Auscultation and olfaction, 3. Inquiry, and 4. Palpation, which are widely used in TCM hospitals in China and around the world. With the development of intelligent computing technology in recent years, computational TCM diagnosis has grown rapidly.
In this paper, we aim to systematically summarize the development of computational TCM diagnosis based on four diagnostic approaches, mainly focusing on digital acquisition devices, collected datasets, and computational detection approaches (algorithms). Furthermore, all related works of this field are compared and explored in detail.
This survey provides the principles, applications, and current progress in computing for readers and researchers in terms of computational TCM diagnosis. Moreover, the future development direction, prospect, and technological trend of computational TCM diagnosis will also be discussed in this study.
Recent computational TCM diagnosis works are compared in detail to show the pros/cons, where we provide some meaningful suggestions and opinions on the future research approaches in this area. This work is useful for disease detection in computational TCM diagnosis as well as health management in the smart healthcare area.
Computational diagnosis, Traditional Chinese Medicine, survey, smart healthcare.
Joint discriminative and collaborative representation for fatty liver disease diagnosis
2017, Expert Systems with Applications
Many people suffer from the Fatty Liver disease due to the changes in diet and lifestyle, and the convenient diagnosis of it has attracted many attentions in recent years. The computerized tongue or facial diagnosis as an important diagnostic tool provides a possible way to detect the disease in the daily life. Most of existing approaches only takes a single modality (e.g., tongue or face) into account, although various modalities would contribute complementary information which is beneficial for the improvement of the diagnosis accuracy. To circumvent this issue, a novel multi-modal fusion method is presented in this paper. Particularly, a noninvasive capture device is first used to captured the tongue and facial images, followed by the feature extraction. Our so-called joint discriminative and collaborative representation approach is then proposed to not only reveal the correlation between the tongue and facial images, but also keep the discriminative representation of each class simultaneously. To optimize the proposed method, an efficient algorithm is proposed, obtaining a closed-form solution and greatly reducing the computation. In identification of the Fatty Liver Disease for healthy controls, the proposed multi-modal fusion approach achieves 85.10% in average accuracy and 0.9363 in the area under ROC curve, which obviously outperform the case of using a single modality and some state-of-the-art methods.
Joint similar and specific learning for diabetes mellitus and impaired glucose regulation detection
2017, Information Sciences
Citation Excerpt :
Five facial blocks were extracted from the facial image to detect hepatitis in [19], and the average accuracy achieved 73.6% using the average RGB pixel intensities features. Zhang and Wang etc. [36] took both color and texture features into account for computerized facial diagnosis. However, despite of various tongue, face or sublingual diagnosis methods proposed for disease detection, most of them regarded either tongue, face or sublingual vein as an independent one and ignored the relationship among them which may have an effect on the overall classification performance.
Effective and accurate diagnosis of Diabetes Mellitus (DM), as well as its early stage Impaired Glucose Regulation (IGR), has attracted much attention recently. Traditional Chinese Medicine (TCM) [Bob Zhang, BVK Kumar, and David Zhang. Detecting diabetes mellitus and nonproliferative diabetic retinopathy using tongue color, texture, and geometry features. Biomedical Engineering, IEEE Transactions on, 61(2):491–501, 2014.], [Bob Zhang, BVK Kumar, and David Zhang. Noninvasive diabetes mellitus detection using facial block color with a sparse representation classifier. Biomedical Engineering, IEEE Transactions on, 61(4):1027–1033, 2014.] etc. has proved that tongue, face and sublingual diagnosis as a noninvasive method is a reasonable way for disease detection. However, most previous works only focus on a single task (tongue, face or sublingual) for diagnosis, although different tasks may provide complementary information for the diagnosis of DM and IGR. In this paper, we propose a novel fusion method to jointly represent the tongue, face and sublingual information and discriminate between DM (or IGR) and healthy controls. Specially, the tongue, facial and sublingual images are first collected by using a non-invasive capture device. The color, texture and geometry features of these three types of images are then extracted, respectively. Finally, our so-called joint similar and specific learning (JSSL) approach is proposed to combine features of tongue, face and sublingual vein, which not only exploits the correlation but also extracts individual components among them. Experimental results on a dataset consisting of 192 Healthy, 198 DM and 114 IGR samples (all samples were obtained from Guangdong Provincial Hospital of Traditional Chinese Medicine) substantiate the effectiveness and superiority of our proposed method for the diagnosis of DM and IGR, achieving 86.07% and 76.68% in average accuracy and 0.8842 and 0.8278 in area under the ROC curves, respectively. The source code can be found in https://github.com/sasky1/JSSLreleased.
Illumination-insensitive texture discrimination based on illumination compensation and enhancement
2014, Information Sciences
Citation Excerpt :
Texture is very important for human visual perception and plays a key role in computer vision and pattern recognition. In addition, since texture can be effectively used for characterizing image regions, texture features have been extensively studied in image classification and content-based image retrieval, as well as in other fields related to pattern analysis [8,9,21,22,38]. Traditionally, texture-representation methods can be divided into three categories, namely structural [34], statistical [17,26], and multi-resolution filtering methods [14,19,24,27–29,36].
As the appearance of a 3D surface texture is strongly dependent on the illumination direction, 3D surface-texture classification methods need to employ multiple training images captured under a variety of illumination conditions for each class. Texture images under different illumination conditions and directions still present a challenge for texture-image retrieval and classification. This paper proposes an efficient method for illumination-insensitive texture discrimination based on illumination compensation and enhancement. Features extracted from an illumination-compensated or -enhanced texture are insensitive to illumination variation; this can improve the performance for texture classification. The proposed scheme learns the average illumination-effect matrix for image representation under changing illumination so as to compensate or enhance images and to eliminate the effect of different and uneven illuminations while retaining the intrinsic properties of the surfaces. The advantage of our method is that the assumption of a single-point light source is not required, so it circumvents and overcomes the limitations of the Lambertian model and is also suitable for outdoor settings. We use a wide range of textures in the PhoTex database in our experiments to evaluate the performance of the proposed method. Experimental results demonstrate the effectiveness of our proposed methods.
An approach to SWIR hyperspectral hand biometrics
2014, Information Sciences
Citation Excerpt :
Hand biometrics is most commonly performed on gray scale images acquired at the visible range (400–700 nm) [7]. Some studies consider the complementarity of three different bands in the visible spectrum, namely the red (650 nm), the green (510 nm) and the blue (475 nm), and combine them at feature, score or decision level [34,20]. These bands are easily obtained with a color camera.
Hand based biometry includes some of the most useful technologies for person identification. The search for new techniques, which complement the battery of existing methods, is an open topic. This paper examines the utility of hyperspectral imagery for hand recognition. Hyperspectral technology permits the sensing of the subsurface tissue structure, which is significantly different from person to person. The data are collected using a SWIR camera in conjunction with an optical spectrograph. This transforms the camera into a line-scan hyperspectral imaging device.
Three feature extraction methods for hyperspectral hand curve characterization are examined. They are based on the area, slope or curvature at different automatically selected spatial hand positions. We report a set of experiments which explore: best hand zones for extracting local hyperspectral features; robustness against the number of training samples; error detection; and occlusion. Different strategies for combining the spectral features with geometric traits available in the hyperspectral cube are discussed. Our experiments show that local spectral properties of human tissue are effective discriminants for biometric recognition with a performance near to or better than that obtained by other hand traits. Equal Error Rates of 0.05% and an identification rate of 96.71% are obtained from a database of 154 people. These results along with their higher robustness to spoofing attacks make the hyperspectral features a promising alternative for person identification.

View all citing articles on Scopus

View full text

Computerized facial diagnosis using both color and texture features

Abstract

Introduction

Section snippets

Facial images and dataset

Facial features extraction

Healthy classification using facial gloss

Facial block-based disease analysis

Conclusions

Acknowledgements

The Foundations of Chinese Medicine

Theoretical Foundations of Chinese Medicine: Systems of Correspondence

Face Reading in Chinese Medicine

Inspection of Face and Body for Diagnosis of Diseases

Computer-aided disease diagnosis system in TCM based on facial image analysis

International Journal of Functional Informatics and Personalized Medicine

Pattern Classification