Skip to main content
SpringerLink
Log in

Skin-based Face Detection-Extraction and Recognition of Facial Expressions

  • Chapter
Applied Pattern Recognition

Part of the book series: Studies in Computational Intelligence ((SCI,volume 91))

Face detection is the foremost task in building vision-based humancomputer interaction systems and in particular in applications such as face recognition, face identification, face tracking, expression recognition and content based image retrieval. A robust face detection system must be able to detect faces irrespective of illuminations, shadows, cluttered backgrounds, facial pose, orientation and facial expressions. Many approaches for face detection have been proposed. However, as revealed by FRVT 2002 tests, face detection in outdoor images with uncontrolled illumination and in images with varied pose (non-frontal profile views) is still a serious problem. In this chapter, we describe a Local-Global Graph (LGG) based method for detecting faces and for recognizing facial expressions accurately in real world image capturing conditions both indoor and outdoor, and with a variety of illuminations (shadows, high-lights, non-white lights) and in cluttered backgrounds. The LG Graph embeds both the local information (the shape of facial feature is stored within the local graph at each node) and the global information (the topology of the face). The LGG approach for detecting faces with maximum confidence from skin segmented images is described. The LGG approach presented here emulates the human visual perception for face detection. In general, humans first extract the most important facial features such as eyes, nose, mouth, etc. and then inter-relate them for face and facial expression representations. Facial expression recognition from the detected face images is obtained by comparing the LG Expression Graphs with the existing the Expression models present in the LGG database. The methodology is accurate for the expression models present in the database.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. M.H. Yang, D.J. Kriegman, and N. Ahuja. Detecting faces in images: A survey. IEEE Pattern Analysis and Machine Intelligence, 24(1), 2002

    Google Scholar 

  2. P. Kakumanu, S. Makrogiannis, R. Bryll, S. Panchanathan, and N. Bourbakis. Image chromatic adaptation using ANNs for skin color adaptation. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence, ICTAI04, 2004

    Google Scholar 

  3. P. Kakumanu. A face and facial expression detection method for visually impaired. Ph.D. Dissertation, Wright State University, Dayton, OH, 2006

    Google Scholar 

  4. A.M. Martinez and R. Benavente. The AR Face Database, CVC Technical Report #24, 1998

    Google Scholar 

  5. X. Yuan and D. Goldman, A. Moghaddamzadeh and N. Bourbakis. Segmentation of colour images with highlights and shadows using fuzzy-like reasoning. Pattern Analysis and Applications, 4(4):272–282, 2001

    Article  MATH  MathSciNet  Google Scholar 

  6. N. Bourbakis, P. Yuan, and S. Makrogiannis. Object recognition using local - global graphs. In Proceedings of the 15thIEEE International Conference on Tools with Artificial Intelligence, ICTAI0, 2003

    Google Scholar 

  7. D. Hearn and M.P. Baker. Computer Graphics, C version. Prentice-Hall, NJ, 1997

    Google Scholar 

  8. N. Ahuja. Dot processing using Voronoi neighborhoods. IEEEPattern Analysis and Machine Intelligence, 4(3), 1982

    Google Scholar 

  9. N. Ahuja, B. An, and B. Schachter. Image representation using Vornoi tesseletation. Computer Vision, Graphics and Image Processing, 29, 1985

    Google Scholar 

  10. K. Arbter, W. E. Snyder, H. Burkhardt and G. Hirzinger. Application of affine invariant Fourier descriptors to recognition of 3D object. IEEEPattern Analysis and Machine Intelligence, 12(7), 1990

    Google Scholar 

  11. P.J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss. The Feret evaluation methodology for face-recognition Algorithms. IEEEPattern Analysis and Machine Intelligence, 22(10), 2000

    Google Scholar 

  12. A.D.J. Cross and E.R. Hancock. Graph matching with a dual step EM algorithm. IEEE Transactions onPattern Analysis and Machine Intelligence, 20(11), 1998

    Google Scholar 

  13. B. Fasel and J. Luettin. Automatic facial expression analysis: A survey. Pattern Recognition, 36:259–275, 2003

    Article  MATH  Google Scholar 

  14. D. Chai and K.N. Ngan. Locating facial region of a head-and-shoulders color image. In ICFGR98, 1998

    Google Scholar 

  15. R. Chellappa, C. Wilson, and S. Sirohey. Human and machine recognition of faces: A survey. Proceedings of IEEE, 83(5):705–740, 1995

    Article  Google Scholar 

  16. A.J. Colmenarez and T.S. Huang. Face detection with information-based maximum discrimination. In Proceedings of CVPR, 1997

    Google Scholar 

  17. T.F. Cootes and C.J. Taylor. Locating faces using statistical feature detectors. In Proceedings of AFGR, pages 204–209, 1996

    Google Scholar 

  18. I. Craw, H. Ellis, and J. Lishman, Automatic extraction of face features. Pattern Recognition Letters, 5:183–187, 1987

    Article  Google Scholar 

  19. I. Craw, D. Tock, and A. Bennett. Finding face features. In Proceedings of the Second European Conference on Computer Vision, pages 92–96, 1992

    Google Scholar 

  20. Y. Dai and Y. Nakano. Face-texture model based on SGLD and its application in face detection in a color scene. Pattern Recognition, 29(6):1007–1017, 1996

    Article  Google Scholar 

  21. J.J. de Dios and N. Garcia. Face detection based on a new color space YCgCr. In ICIP03, 2003

    Google Scholar 

  22. B.A. Draper, K. Baek, M.S. Bartlett, and J.R. Beveridge. Recognizing faces with PCA and ICA. Computer Vision Image Understanding, 91(1–2):115–137, 2003

    Article  Google Scholar 

  23. G.J. Edwards, C.J. Taylor, and T. Cootes. Learning to Identify and Track Faces in Image Sequences. In Proceedings of ICCV, pages 317–322, 1998

    Google Scholar 

  24. P. Ekman and W. Frisen. Facial Action Coding System, Palo Alto, CA. Consulting Psychologists Press, 1978

    Google Scholar 

  25. R. Fe’raud, O.J. Bernier, J.-E. Villet, and M. Collobert. A fast and accurate face detector based on neural networks. Pattern Analysis and Machine Intelligence, 22(1): 42–53, 2001

    Google Scholar 

  26. C. Garcia and G. Tziritas. Face detection using quantized skin color regions merging and wavelet packet analysis. IEEE Transactions on Multimedia, 1(3):264–277, 1999

    Article  Google Scholar 

  27. V. Govindaraju. Locating human faces in photographs. International Journal of Computer Vision, 19(2):129–146, 1996

    Article  MathSciNet  Google Scholar 

  28. V. Govindaraju, S.N. Srihari, and D.B. Sher. A computational model for face location. In Proceedings of the International Conference on Computer Vision, pages 718–721, 1990

    Google Scholar 

  29. E. Hjelmas and B.K. Low. Face detection: A survey. Journal of Computer Vision and Image Understanding, 83:236–274, 2001

    Article  MATH  Google Scholar 

  30. R.L. Hsu, M. Abdel-Mottaleb, and A.K. Jain. Face detection in color images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5):696–706, 2002

    Article  Google Scholar 

  31. S.H. Kim, N.K. Kim, S.C. Ahn, and H.G. Kim. Object oriented face detection using range and color information. In AFGR98, 1998

    Google Scholar 

  32. M. Kirby and L. Sirovich. Application of the Karhunen–Loe’ve procedure for the characterization of human faces. Pattern Analysis and Machine Intelligence, 12(1), 1990

    Google Scholar 

  33. Y.H. Kwon and N. da Vitoria Lobo. Face detection using templates. In Proceedings of ICPR, pages 764–767, 1994

    Google Scholar 

  34. S.G. Kong, J. Heo, B.R. Abidi, J. Paik, and M.A. Abidi. Recent advances in visual and infrared face recognition – a review. Computer Vision and Image Understanding, 97, 2005

    Google Scholar 

  35. J. Kovac, P. Peer, and F. Solina. Human skin color clustering for face detection. In EUROCON2003, 2003

    Google Scholar 

  36. C. Kotropoulos, A. Tefas, and I. Pitas, Frontal face authentication using morphological elastic graph matching. IEEE Transactions on Image Processing, 9(4):555–560, 2000

    Article  Google Scholar 

  37. P. Kuchi, P. Gabbur, S. Bhat, and S. David. Human face detection and tracking using skin color modeling and connected component operators. IETE Journal of Research, Special issue on Visual Media Processing, 2002

    Google Scholar 

  38. V. Kumar and T. Poggio. Learning-based approach to real time tracking and analysis of faces. In Proceedings of AFGR, 2000

    Google Scholar 

  39. A. Lanitis, C.J. Taylor, and T.F. Cootes. An automatic face identification system using flexible appearance models. Image and Vision Computing 13(5):393–401, 1995

    Article  Google Scholar 

  40. M.S. Lew and N. Huijsmans. Information theory and face detection. In Proceedings of ICPR, 1996

    Google Scholar 

  41. C. Liu and H. Wechsler. Comparative assessment of independent component analysis (ICA) for face recognition. In Proceedings of the Second International Conference on Audio- and Video-based Biometric Person Authentication, Washington, DC, 1999

    Google Scholar 

  42. S. Mann. Wearable, tetherless computer-mediated reality: wearcam as a wearable face-recognizer, and other applications for the disabled. Technical Report TR 361, MIT Media Lab Perceptual Computing Section, Cambridge, MA, 1996. Also, available at http://www.eyetap.org/

  43. F. Marqués and V. Vilaplana. A morphological approach for segmentation and tracking of human face. In ICPR 2000, 2000

    Google Scholar 

  44. S. McKenna, S. Gong, and Y. Raja. Modeling facial colour and identity with Gaussian mixtures. Pattern Recognition, 31(12):1883–1892, 1998

    Article  Google Scholar 

  45. L. Meng and T. Nguyen. Two subspace methods to discriminate faces and clutters. In Proceedings of ICIP, 2000

    Google Scholar 

  46. J. Miao, B. Yin, K. Wang, L. Shen, and X. Chen. A hierarchical multiscale and multiangle system for human face detection in a complex background using gravity-center template. Pattern Recognition, 32(7):1237–1248, 1999

    Article  Google Scholar 

  47. A.V. Nefian and M. H. Hayes III. Face detection and recognition using hidden Markov models. In Proceedings ofICIP, 1:141–145, 1998

    Google Scholar 

  48. K. Okada, J. Steffens, T. Maurer, H. Hong, E. Elagin, H. Neven, and C. von der Malsburg. The Bochum/USC face recognition system and how it fared in the Feret phase III test. In Face Recognition: From Theory to Applications. Springer, Berlin Heidelberg New York, 1998

    Google Scholar 

  49. N. Oliver, A. Pentland, and F. Berard. Lafter: Lips and face real time tracker. In CVPR97, 1997

    Google Scholar 

  50. E. Osuna, R. Freund, and F. Girosi. Training support vector machines: An application to face detection. In Proceedings of CVPR, pages 130–136, 1997

    Google Scholar 

  51. Z. Pan, G. Healey, M. Prasad, and B. Tromberg. Face recognition in hyperspectral images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 2003

    Google Scholar 

  52. A. Pentland, B. Moghaddam, and T. Starner. View-based and modular eigenspaces for face recognition. In Proceedings of IEEE International Conference CVPR, pages 84–91, 1994

    Google Scholar 

  53. P.J. Phillips, P. Grother, R.J. Micheals, D.M. Blackburn, E. Tabassi, M. Bone. Face recognition vendor test: Evaluation report, 2003

    Google Scholar 

  54. S.L. Phung, A. Bouzerdoum, and D. Chai. A novel skin color model in YCBCR color space and its application to human face detection. In ICIP02, 2002

    Google Scholar 

  55. M. Propp and A. Samal. Artificial neural network architecture for human face detection. Intelligent Engineering Systems Through Artificial Neural Networks, 2:535–540, 1992

    Google Scholar 

  56. D. Roth, M.-H. Yang, and N. Ahuja, A SNoW-based face detector. In NIPS, volume 12. MIT, Cambridge, MA, 2000

    Google Scholar 

  57. H. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection. In CVPR, pages 203–208, 1996

    Google Scholar 

  58. H. Rowley, S. Baluja, and T. Kanade, Neural network-based face detection. Pattern Analysis and Machine Intelligence, 20(1):23–38, 1998

    Article  Google Scholar 

  59. H. Rowley, S. Baluja, and T. Kanade, Rotation invariant neural network-based face detection. In Proceedings of CVPR, pages 38–44, 1998

    Google Scholar 

  60. E. Saber and A.M. Tekalp, Frontal-view face detection and facial feature extraction using color, shape and symmetry based cost functions. Pattern Recognition Letters, 17(8), 1998

    Google Scholar 

  61. H. Sahbi and N. Boujemaa. Coarse to fine face detection based on skin color adaptation. In Workshop on Biometric Authentication, 2002, volume 2359, LNCS, pages 112–120, 2002

    Google Scholar 

  62. F. Samaria and S. Young. HMM based architecture for face identification. Image and Vision Computing, 12:537–583, 1994

    Article  Google Scholar 

  63. A. Samal and P.A. Iyengar. Human face detection using silhouettes. International Journal of Pattern Recognition and Artificial Intelligence 9(6), 1995

    Google Scholar 

  64. H. Schneiderman and T. Kanade. A statistical method for 3D object detection applied to faces and cars. In Proceedings of CVPR, volume 1, pages 746–751, 2000

    Google Scholar 

  65. K. Schwerdt and J.L. Crowely. Robust face tracking using color. In AFGR00, 2000

    Google Scholar 

  66. D.A. Socolinsky, A. Selinger, and J.D. Neuheisel. Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding, 91(1–2):72–114, 2003

    Article  Google Scholar 

  67. K. Sobottka and I. Pitas, Extraction of facial regions and features using color and shape information. In ICPR96, 1996

    Google Scholar 

  68. K. Sobottka and I. Pitas. A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Processing: Image Communication, 12:263–281, 1998

    Article  Google Scholar 

  69. F. Soulie, E. Viennet, and B. Lamy. Multi-modular neural network architectures: Pattern recognition applications in optical character recognition and human face recognition. International Journal of Pattern Recognition and Artificial Intelligence, 7(4):721–755, 1993

    Article  Google Scholar 

  70. S. Srisuk and W. Kurutach. New robust face detection in color images. In AFGR02, pages 291–296, 2002

    Google Scholar 

  71. K.-K. Sung and T. Poggio. Example-based learning for view-based human face detection. Pattern Analysis and Machine Intelligence, 20, 1998

    Google Scholar 

  72. A. Tefas, C. Kotropoulos, and I. Pitas. Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication. Pattern Analysis and Machine Intelligence, 23(7):735–746, 2001

    Article  Google Scholar 

  73. J. Terrillon, M. Shirazi, M. Sadek, H. Fukamachi, and S. Akamatsu. Invariant face detection with support vector machines. In Proceedings of ICPR, 2000

    Google Scholar 

  74. M. Turk and A. Pentland. Face recognition using eigenfaces. In Proceedings of CVPR, pages 586–591, 1991

    Google Scholar 

  75. J.G. Wang and E. Sung. Frontal-view face detection and facial feature extraction using color and morphological operations.Pattern Recognition Letters, 20:1053–1068, 1999

    Article  Google Scholar 

  76. Y. Wang and B. Yuan. A novel approach for human face detection from color images under complex background. Pattern Recognition, 34(10):1983–1992, 2001

    Article  MATH  MathSciNet  Google Scholar 

  77. L. Wiskott and C. von der Malsburg. Recognizing faces by dynamic link matching. Neuroimage, 4(3):S14–S18, 1996

    Article  Google Scholar 

  78. K.W. Wong, K.M. Lam, and W.C. Siu. A robust scheme for live detection of human faces in color images. Signal Processing: Image Communication, 18(2):103–114, 2003

    Article  Google Scholar 

  79. M.H. Yang and N. Ahuja. Detecting human faces in color images. In ICIP98, 1998

    Google Scholar 

  80. M.-H. Yang, N. Ahuja, and D. Kriegman. Face detection using mixtures of linear subspaces. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000

    Google Scholar 

  81. T.W. Yoo and I.S. Oh. A fast algorithm for tracking human faces based on chromatic histograms. Pattern Recognition Letters, 20(10):967–978, 1999

    Article  Google Scholar 

  82. A. Yuille, P. Hallinan, and D. Cohen, Feature extraction from faces using deformable templates. International Journal of Computer Vision, 8(2):9–111, 1992

    Article  Google Scholar 

  83. W. Zhao, R. Chellappa, P.J. Philips, and A. Rosenfeld, Face recognition: A literature survey, ACM Computing Surveys, 85(4):299–458, 2003

    Google Scholar 

  84. X. Zhang, Y. Jia, A linear discriminant analysis framework based on random subspace for face recognition. Pattern Recognition, 40:2585–2591, 2007

    Article  MATH  Google Scholar 

  85. J. Meyneta, V. Popovicib, and J.-P. Thirana, Face detection with boosted Gaussian features, Pattern Recognition 40, 2007

    Google Scholar 

  86. Q. Chen.,W.-k. Cham, and K.-k. Lee, Extracting eyebrowcontour and chin contour for face recognition, Pattern Recognition, 40, 2007

    Google Scholar 

  87. S.-I. Choi, C. Kim, and C.-H. Choi. Shadowcompensation in 2D images for face recognition. Pattern Recognition, 40, 2007

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Technology Research Institute, Wright State University, 45435, Dayton, OH, USA

    N. Bourbakis & P. Kakumanu

Authors
  1. N. Bourbakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Kakumanu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Science and Applied Mathematics, Neubrückstrasse 10, CH-3012, Bern, Switzerland

    Horst Bunke

  2. University of South Florida, 4202 E. Fowler Ave, 33620, Tampa, FL, USA

    Abraham Kandel

  3. Ben-Gurion University of the Negev, 84105, Beer-Sheva, Israel

    Mark Last

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Bourbakis, N., Kakumanu, P. (2008). Skin-based Face Detection-Extraction and Recognition of Facial Expressions. In: Bunke, H., Kandel, A., Last, M. (eds) Applied Pattern Recognition. Studies in Computational Intelligence, vol 91. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76831-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-76831-9_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-76830-2

  • Online ISBN: 978-3-540-76831-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation