Physics-based modelling of human skin colour under mixed illuminants

https://doi.org/10.1016/S0921-8890(01)00122-1Get rights and content

Abstract

Skin colour is an often used feature in human face and motion tracking. It has the advantages of being orientation and size invariant and it is fast to process. The major disadvantage is that it becomes unreliable if the illumination changes. In this paper, skin colour is modelled based on a reflectance model of the skin, the parameters of the camera and light sources. In particular, the location of the skin colour area in the chromaticity plane is modelled for different and mixed light sources. The model is empirically validated. It has applications in adaptive segmentation of skin colour and in the estimation of the current illumination in camera images containing skin colour.

Introduction

There are many computer vision applications which automatically detect and track human faces and human motion. These applications are as diverse as: human–computer interfaces, surveillance systems, and automatic camera men.

Skin colour is an often used feature in such systems [5], [6], [10], [11], [12], [13]. It is mostly used as an initial approximate localisation and segmentation of faces or hands in the camera image. This reduces the search region for other more precise and computationally expensive facial feature detection methods. Thus, it is important that the output of the skin colour segmentation gives reliable information. Advantages of using skin colour are that it is size and orientation invariant, and fast to process. It is suitable for real time systems [6], [13].

A problem with robust skin colour detection arises under varying lighting conditions. The same skin area appears as two different colours under two different illuminations. This is a well known problem in colour vision and there are several colour constancy methods trying to cope with this. However, until now no generic method has been found [8].

The appearance of skin depends on the brightness and the colour of the illumination. The dependency on the brightness can be resolved by transforming into a chromatic colour space as done by, e.g., Schiele and Waibel [13]. McKenna et al. [10] report a face tracking system where the skin colour segmentation algorithm adapts to the skin chromaticity distribution using adaptive Gaussian mixture models. They can track a face moving relative to fixed light sources, i.e. they can cope with changing illumination geometry, but no tests under significant changes of the illuminant colour are reported.

The dependency on the illuminant colour is usually not taken into account in skin colour segmentation, despite the fact that the skin colour chromaticities may change drastically when changing the illumination colour. Only recently the changes of skin colour under different illumination colours were investigated [16], [17]. Störring et al. [17] show that it is possible to model skin colour for different illuminant colours with a good approximation.

This paper investigates theoretically and experimentally the colour image formation process, in particular with respect to human skin colour, using one or two illumination sources.

We first review general models for reflection and for illumination sources. Then, the reflections of skin colour are modelled. A series of experiments is done in order to compare the measured with the modelled data.

The specific problems addressed are:

  • How well can skin colour reflections be modelled for a range of illumination sources and for a range of different humans with varying skin colours due their genetic backgrounds.

  • How well does the modelling of illumination sources by Blackbody radiators apply in this application context.

  • How well does the modelling of mixed illumination sources apply for skin colour analysis.

The answer to the first problem provides insight about the potential of using skin colour as a general robust method in a range of computer vision applications when humans appear in the scenario. The second problem evaluates the possibilities of using simple models for frequently used illumination sources. The last problem is extremely important for the practical applications of skin colour analysis, as mixed illumination is a typical situation in every day live.

Section snippets

Modelling colour reflections and light sources

The image formation process using a colour video camera can be modelled by spectral integration. Knowing the spectrum of the incoming light, the spectral sensitivities of the camera’s sensing elements, and the spectral transmittance of filters in front of the sensing elements one can model, e.g., the red, green, and blue (RGB) pixel values. The incoming light results from the spectrum of the light source and the reflectance characteristic of the reflecting material. This is often described by

Reflectance of human skin

Skin is composed of a thin surface layer, the epidermis, and the dermis, which is a thicker layer placed under the epidermis. Surface reflection of skin takes place at the epidermis surface. It is approximately ρsurf=5% of the incident light independent of its wavelength [3]. The rest of the incident light (95%) is entering the skin where it is absorbed and scattered within the two skin layers and eventually reflected (body reflectance). The epidermis mainly absorbs light, it has the properties

Image acquisition

For the experimental test of the skin colour model images are taken from several faces under different illuminations.

The images are captured with a JAI CV-M90 3CCD colour video camera equipped with a Fudjinon wide angle lens and connected to a Silicon Graphics Sirius or a FlashBus Integral Technologies frame-grabber. No automatic gain control or automatic white balancing are used and the gamma correction is set to one. The lens opening and the shutter speed are manually adjusted to make use of

Comparison of measured and modelled data

The images were hand segmented into skin areas and non-skin areas. The skin areas are used as skin colour measurements. First images taken under a single illumination colour will be investigated for known illumination spectra and for blackbody radiator modelled illumination. At the end of this section images taken under mixed illumination will be discussed.

Conclusions

In the previous sections it was shown that it is possible to model human skin colour under changing and mixed illuminations. Indoor images of eight people with different skin colours under four different CCTs and mixtures of these confirm this. Body reflections of human skin are modelled by spectral integration using the spectral sensitivities of the camera, spectral reflectance curves of human skin, and the relative spectral radiance of the light sources. The spectral reflectance curves of

Acknowledgements

We would like to thank all the persons who participated in the image acquisition, and would like to thank the European Communities TMR research network SMART II, Contract No. ERBFMRX-CT96-0052, for supporting this research.

Moritz Störring studied Electronical Engineering at the Technical University of Berlin, Germany, and at the Institut National Polytechnique de Grenoble, France. He graduated in 1998 in Berlin. Since April 1998, he has been employed as a Research Assistant at the Computer Vision and Media Technology Laboratory, Aalborg University, Denmark, within the European TMR project SMART II. His research interests include colour vision, outdoor computer vision, face detection, and vision based

References (20)

  • OSRAM Lichtprogramm, Product Catalogue,...
  • H.J. Andersen et al.

    Classifying illumination condition from two light sources by colour histogram assessment

    Journal of the Optical Society of America A

    (2000)
  • R.R. Anderson J. Hu, J.A. Parrish. Optical radiation transfer in the human skin and applications in in vivo remittance...
  • Anonymous. Method of measuring and specifying colour rendering properties of light sources, Technical Report CIE 13.3,...
  • C. Chen, S.-P. Chiang, Detection of human faces in colour images, IEE Proceedings on Vision Image and Signal Processing...
  • J.L. Crowley et al.

    Things that see

    Communications of the ACM

    (2000)
  • G. Finlayson, G. Schaefer, Single surface colour constancy, in: Proceedings of the Seventh Color Imaging Conference,...
  • B. Funt, K. Barnard, L. Martin, Is machine colour constancy good enough, in: H. Burkhardt, B. Neumann (Eds.),...
  • G.J. Klinker et al.

    A physical approach to color image understanding

    International Journal of Computer Vision

    (1990)
  • S.J. McKenna et al.

    Modelling facial colour and identity with Gaussian mixtures

    Pattern Recognition

    (1998)
There are more references available in the full text version of this article.

Cited by (47)

  • Near infrared face recognition by combining Zernike moments and undecimated discrete wavelet transform

    2014, Digital Signal Processing: A Review Journal
    Citation Excerpt :

    Most of them have focused on visible face recognition due to the fact that face recognition is one of the primary activities of the human visual system. The main limitation of visible face recognition, however, is the high dependency of system performance on external light, angle of light and even skin color [6,7]. Various methods have been developed and introduced to solve the illumination problem by proposing illumination invariant face recognition [8–10].

  • Real-time detection of children's skin on social networking sites using Markov random field modelling

    2011, Information Security Technical Report
    Citation Excerpt :

    Cai and Goshtasby (1999) and Kawato and Ohya (2000) have employed CIE-Lab and (Yang and Ahuja, 1999) employed CIE-Luv in their skin detectors. Further, a novel human skin color model under changed and mixed lighting conditions has been demonstrated by Strring et al. (30 June 2001), based on a reflectance model of the skin, the parameters of the cameras and the light sources with a special emphasis on modeling the location of the skin color area in the chromaticity plane under different and mixed illumination sources. At this stage, we cannot make any general conclusion on either suitability or preference of any particular color space.

  • Skin detection for single images using dynamic skin color modeling

    2010, Pattern Recognition
    Citation Excerpt :

    Its application includes face detection and recognition, gesture analysis, image content classification, etc. Earlier studies [1–3] have shown that human skin color generally clusters within a certain range in the color space and can be used to identify skin pixels in an image. Skin detection methods reported in the literature include explicit skin-color thresholding, Bayes classifiers, neural network classifiers, maximum entropy classifiers, and so on [4].

  • Task-based evaluation of skin detection for communication and perceptual interfaces

    2007, Journal of Visual Communication and Image Representation
  • Symbolic factorial discriminant analysis for illumination invariant face recognition

    2008, International Journal of Pattern Recognition and Artificial Intelligence
View all citing articles on Scopus

Moritz Störring studied Electronical Engineering at the Technical University of Berlin, Germany, and at the Institut National Polytechnique de Grenoble, France. He graduated in 1998 in Berlin. Since April 1998, he has been employed as a Research Assistant at the Computer Vision and Media Technology Laboratory, Aalborg University, Denmark, within the European TMR project SMART II. His research interests include colour vision, outdoor computer vision, face detection, and vision based human–computer interaction. He is currently working towards a Ph.D. on human skin colour modelling and detection.

Hans Jørgen Andersen has received a Ph.D. degree from the Computer Vision and Media Technology Laboratory, Aalborg University, 2001, within the field Outdoor Computer Vision. Prior to his Ph.D., he has received a B.Sc. in Mechanical Engineering from Horsens Technical College, 1990, and worked as a Research Assistant at the Norwegian Centre for Ecological Agriculture. After his Ph.D., he has worked as Research and Development Manager at Eco-Dan A/S introducing the first commercial system for computer vision supported guidance of agricultural machinery. Currently, he is with the Technology and Innovation Department at Siemens Mobile Phones, Pandrup, Denmark.

Erik Granum is Professor of Information Systems and Head of CVMT, Computer Vision and Media Technology Laboratory (Lab. of Image Analysis), at Aalborg University, Denmark, where he came in 1983. He got his B.Sc. in 1967, worked in industry, M.Sc. in 1973, Ph.D. in 1981, and worked for the British MRC in Edinburgh. He has been co-ordinator of several national and international research projects and networks in computer vision (including VAP, Vision as Process), and partner of many others. He is founding member of I3NET, a main partner of projects on Virtual Theatre and Visual Data Mining, partner of EU-network VIRGO, and co-ordinator of EU-project PUPPET, the “Educational Puppet Theatre of Virtual Worlds”. He was a main actor in the establishment of a multimedia and virtual reality centre at Aalborg University. His research interests cover pattern recognition, motion analysis, continually operating vision systems, colour vision, vision guided multimedia interfaces, visualisation, virtual reality, and autonomous agents.

View full text